[Gluster-devel] Gluster Coreutils

Mon Jun 15 19:19:27 UTC 2015

Hey all,

I'm Craig's manager for the duration of his internship @ FB, so I thought I'd better chime in here :).  As Craig mentioned, our project plan is to implement C-based CLI utilities similar to what we have for NFS CLI utilities (we'll be open sourcing this in the coming days so you can see what we have there).  Our time to completion is very aggressive for a project coded in C (~10 weeks or less), so we need to keep our project goals very focused (feature creep, architecture rabbit holes etc) to keep it on schedule and come out with a working "product" that scales to our use-cases.

As such, rather than have "too many cook in the kitchen", I'd propose the initial implementation of ls/tail/cat/put/mkdir/rm/stat/cp be implemented by Craig, and we'll put the code up on the public repo as early as possible for public review and comment.  We've definitely heard some great ideas from the list, and we've changed our approach to include:

1. Shell based interaction
2. Single binary (behavior changes based on symlink; any cons to this?)
3. Separate code repo/packaging (we'll probably go C99 because of this since we are no longer bound by the legacy C89 standards of the GFS repo).

On the language debate (Python vs Go vs C), rather than get into this debate (which has good arguments on all sides) I'd simply encourage anyone who feels strongly about implementing the CLI in a different language to simply do so!  More choice and competition among tools is never a bad thing, as it will only make them both better.  Suffice it to say though, we feel strongly about the C based implementation so we are taking it on to provide parity with our NFS CLI utilities (which I'm working to get open-sourced in the coming weeks).

And finally, with respect to scaling: we feel reasonably confident we can scale the tools without any sort of proxy to work on the order of 100's of simultaneous CLI clients hitting a cluster.  We are going to experiment with some "proxy" approaches others to try to scale this to 1000's of CLI clients into a single cluster.  Though, the proxy will be optional as we feel the simplicity/reliability of a standalone client (hitting a standard load-balanced or DNS RR end point) will work well for vast majority of users (envision a "-p <proxy>" option in the tools for those who need to really scale them).

Richard

________________________________________
From: gluster-devel-bounces at gluster.org [gluster-devel-bounces at gluster.org] on behalf of Craig Cabrey [craigcabrey at fb.com]
Sent: Monday, June 15, 2015 10:50 AM
To: Poornima Gurusiddaiah
Cc: gluster-devel at gluster.org
Subject: Re: [Gluster-devel] Gluster Coreutils

The initial versions of the commands will establish independent connections to a cluster node (specified by a Gluster URL scheme). Then I think I could evolve the code to implement the shell idea that was in the design doc. Then, if I have time left, I could try to tackle the daemon that would keep connections open (whether this would be a client side or a server side proxy is yet to be determined). I'm leaving that for the end since that would require the most design and I don't want to end up going down a rabbit hole at this point trying to fight DBUS, for example.

I have been reading through the document posted earlier to get a sense of the overall direction the project should move in. As for the language, I already have functioning code in C and that's a requirement of my project, so that's what I'm moving forward with.

Craig

> On Jun 15, 2015, at 12:04 AM, Poornima Gurusiddaiah <pgurusid at redhat.com> wrote:
>
> Hi Craig,
>
> That's cool! I was more interested in knowing how you plan to implement the commands.
> To be specific, do you plan to connect/disconnect(glfs_init/glfs_fini) to the gluster
> server for each command or persist the connection across commands?
>
> Regards,
> Poornima
>
> ----- Original Message -----
>> From: "Craig Cabrey" <craigcabrey at fb.com>
>> To: "Joe Julian" <joe at julianfamily.org>
>> Cc: gluster-devel at gluster.org
>> Sent: Monday, June 15, 2015 3:19:07 AM
>> Subject: Re: [Gluster-devel] Gluster Coreutils
>>
>> I've already started writing the utilities in C per my internship project.
>> I'll push these up when ready (most probably sometime this week) as a POC.
>>
>> Maybe then we can look into implementing with Python?
>>
>> Craig
>>
>>> On Jun 14, 2015, at 2:47 PM, Joe Julian <joe at julianfamily.org> wrote:
>>>
>>> I was thinking the other way around. Write it in python then optimize if
>>> it's necessary.
>>>> On 06/14/2015 02:45 PM, chris holcombe wrote:
>>>> Maybe we could write these in C and setup python bindings for them.
>>>> Thoughts?  I'm down with writing them in C.  I could use more practice.
>>>>
>>>>> On 06/14/2015 02:36 PM, Joe Julian wrote:
>>>>> I would prefer python.
>>>>>
>>>>>> On 06/14/2015 11:18 AM, Niels de Vos wrote:
>>>>>>> On Sat, Jun 13, 2015 at 06:45:45PM +0530, M S Vishwanath Bhat wrote:
>>>>>>> On 12 June 2015 at 23:59, chris holcombe <chris.holcombe at canonical.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yeah I have this repo but it's basically empty:
>>>>>>>> https://github.com/cholcombe973/GlusterUtils
>>>>>>>
>>>>>>> AFAIK the plan is to collaborate through a git repo in
>>>>>>> github.com/gluster
>>>>>>> account. But anything that works should be good...
>>>>>>>
>>>>>>> And the choice of language seems to be python.
>>>>>> Depending on the target systems, Python may not be suitable. There are
>>>>>> cloud deployments that do not have Python installed. I think that even
>>>>>> includes the minimal cloud images Fedora and CentOS provide. IMHO would
>>>>>> be nice to be able to support those systems too, without pulling in too
>>>>>> many dependencies.
>>>>>>
>>>>>> But, I'll leave it up to the people writing the code ;-)
>>>>>>
>>>>>> Cheers,
>>>>>> Niels
>>>>>>
>>>>>>> Best Regards,
>>>>>>> Vishwanath
>>>>>>>
>>>>>>>>
>>>>>>>>> On 06/12/2015 11:27 AM, Craig Cabrey wrote:
>>>>>>>>>
>>>>>>>>> Chris,
>>>>>>>>>
>>>>>>>>> That sounds good to me.
>>>>>>>>>
>>>>>>>>> I already have started on implementation, just to get familiar with
>>>>>>>>> the
>>>>>>>>> codebase and GFAPI.
>>>>>>>>>
>>>>>>>>> Is there a public repo that we can use for collaboration?
>>>>>>>>>
>>>>>>>>> Craig
>>>>>>>>>
>>>>>>>>> On Jun 12, 2015, at 10:46 AM, chris holcombe <
>>>>>>>>>> chris.holcombe at canonical.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Craig,
>>>>>>>>>>
>>>>>>>>>> I was actually planning on building the same tool set. I would like
>>>>>>>>>> to
>>>>>>>>>> work with you also on this if that's ok.
>>>>>>>>>>
>>>>>>>>>> -Chris
>>>>>>>>>>
>>>>>>>>>>> On 06/12/2015 10:43 AM, Jeff Darcy wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>> This summer I am an intern at Facebook working on the Gluster
>>>>>>>>>>>> team.
>>>>>>>>>>>> Part of
>>>>>>>>>>>> my project for the summer includes developing a set of coreutils
>>>>>>>>>>>> that
>>>>>>>>>>>> utilizes the Gluster C API natively.
>>>>>>>>>>>>
>>>>>>>>>>>> This project is similar in nature to the NFS coreutils that some
>>>>>>>>>>>> of
>>>>>>>>>>>> you may
>>>>>>>>>>>> have heard about from the other Facebook engineers at the Gluster
>>>>>>>>>>>> summit
>>>>>>>>>>>> recently. I just wanted to reach out to the Gluster community to
>>>>>>>>>>>> gather
>>>>>>>>>>>> ideas, potential features, feedback, and direction.
>>>>>>>>>>>>
>>>>>>>>>>>> The initial set of utilities that I am developing includes the
>>>>>>>>>>>> following:
>>>>>>>>>>>>
>>>>>>>>>>>> * cat
>>>>>>>>>>>> * mkdir
>>>>>>>>>>>> * put (read from stdin and write to a file)
>>>>>>>>>>>> * mv
>>>>>>>>>>>> * ls
>>>>>>>>>>>> * rm
>>>>>>>>>>>> * tail
>>>>>>>>>>>>
>>>>>>>>>>>> Again, any feedback will be welcome.
>>>>>>>>>>> Hi, Craig, and welcome to the project.  :)
>>>>>>>>>>>
>>>>>>>>>>> There seems to be some overlap with a proposal Ragahavendra Talur
>>>>>>>>>>> sent
>>>>>>>>>>> out
>>>>>>>>>>> a couple of days ago.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=https://docs.google.com/document/d/1yuRLRbdccx_0V0UDAxqWbz4g983q5inuINHgM1YO040/edit?usp%3Dsharing&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0A&s=8d55bb5770b8ed1d683a6908a05af32b79289735c537c660252fcaa7c690e162
>>>>>>>>>>>
>>>>>>>>>>> This seems like an excellent opportunity to collaborate.  Ideally,
>>>>>>>>>>> I
>>>>>>>>>>> think
>>>>>>>>>>> it would be useful to have both an FTP-client-like "shell" and a
>>>>>>>>>>> set of
>>>>>>>>>>> standalone "one shot" commands, based on as much common code as
>>>>>>>>>>> possible.
>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Gluster-devel mailing list
>>>>>>>>>>> Gluster-devel at gluster.org
>>>>>>>>>>>
>>>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0A&s=28546cdc6fdf6f75f4cfa4b8260abc595eee96601a5f849ebb230ddbd1faf8b3
>>>>>>>> _______________________________________________
>>>>>>>> Gluster-devel mailing list
>>>>>>>> Gluster-devel at gluster.org
>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>>>> _______________________________________________
>>>>>>> Gluster-devel mailing list
>>>>>>> Gluster-devel at gluster.org
>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>>> _______________________________________________
>>>>>> Gluster-devel mailing list
>>>>>> Gluster-devel at gluster.org
>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>>
>>>>> _______________________________________________
>>>>> Gluster-devel mailing list
>>>>> Gluster-devel at gluster.org
>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org
>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=TAgCUd2QFR665sNN6ukXRFvoAv6uRiOVHk%2Bfk5BIhO8%3D%0A&s=8a2325c4cc557cc546ce9432f9f7055efa6f6af5d351046a3f230d63aa98a7c0
>>

_______________________________________________
Gluster-devel mailing list
Gluster-devel at gluster.org
https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=eSXH2j44tGdWnHEbaIk1Tg%3D%3D%0A&m=Wd%2FBeWxuvw7AE0esMcPOM4ru3joVrMtHoFVK7u2QXIQ%3D%0A&s=8ed0d55446f6fbddfea3f76c1bb09633b7d0bdabeef9b08d389320ab8a7e1291