[Gluster-devel] Gluster Coreutils

Mon Jun 15 17:50:19 UTC 2015

The initial versions of the commands will establish independent connections to a cluster node (specified by a Gluster URL scheme). Then I think I could evolve the code to implement the shell idea that was in the design doc. Then, if I have time left, I could try to tackle the daemon that would keep connections open (whether this would be a client side or a server side proxy is yet to be determined). I'm leaving that for the end since that would require the most design and I don't want to end up going down a rabbit hole at this point trying to fight DBUS, for example.

I have been reading through the document posted earlier to get a sense of the overall direction the project should move in. As for the language, I already have functioning code in C and that's a requirement of my project, so that's what I'm moving forward with.

Craig

> On Jun 15, 2015, at 12:04 AM, Poornima Gurusiddaiah <pgurusid at redhat.com> wrote:
> 
> Hi Craig,
> 
> That's cool! I was more interested in knowing how you plan to implement the commands.
> To be specific, do you plan to connect/disconnect(glfs_init/glfs_fini) to the gluster
> server for each command or persist the connection across commands?
> 
> Regards,
> Poornima
> 
> ----- Original Message -----
>> From: "Craig Cabrey" <craigcabrey at fb.com>
>> To: "Joe Julian" <joe at julianfamily.org>
>> Cc: gluster-devel at gluster.org
>> Sent: Monday, June 15, 2015 3:19:07 AM
>> Subject: Re: [Gluster-devel] Gluster Coreutils
>> 
>> I've already started writing the utilities in C per my internship project.
>> I'll push these up when ready (most probably sometime this week) as a POC.
>> 
>> Maybe then we can look into implementing with Python?
>> 
>> Craig
>> 
>>> On Jun 14, 2015, at 2:47 PM, Joe Julian <joe at julianfamily.org> wrote:
>>> 
>>> I was thinking the other way around. Write it in python then optimize if
>>> it's necessary.
>>>> On 06/14/2015 02:45 PM, chris holcombe wrote:
>>>> Maybe we could write these in C and setup python bindings for them.
>>>> Thoughts?  I'm down with writing them in C.  I could use more practice.
>>>> 
>>>>> On 06/14/2015 02:36 PM, Joe Julian wrote:
>>>>> I would prefer python.
>>>>> 
>>>>>> On 06/14/2015 11:18 AM, Niels de Vos wrote:
>>>>>>> On Sat, Jun 13, 2015 at 06:45:45PM +0530, M S Vishwanath Bhat wrote:
>>>>>>> On 12 June 2015 at 23:59, chris holcombe <chris.holcombe at canonical.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Yeah I have this repo but it's basically empty:
>>>>>>>> https://github.com/cholcombe973/GlusterUtils
>>>>>>> 
>>>>>>> AFAIK the plan is to collaborate through a git repo in
>>>>>>> github.com/gluster
>>>>>>> account. But anything that works should be good...
>>>>>>> 
>>>>>>> And the choice of language seems to be python.
>>>>>> Depending on the target systems, Python may not be suitable. There are
>>>>>> cloud deployments that do not have Python installed. I think that even
>>>>>> includes the minimal cloud images Fedora and CentOS provide. IMHO would
>>>>>> be nice to be able to support those systems too, without pulling in too
>>>>>> many dependencies.
>>>>>> 
>>>>>> But, I'll leave it up to the people writing the code ;-)
>>>>>> 
>>>>>> Cheers,
>>>>>> Niels
>>>>>> 
>>>>>>> Best Regards,
>>>>>>> Vishwanath
>>>>>>> 
>>>>>>>> 
>>>>>>>>> On 06/12/2015 11:27 AM, Craig Cabrey wrote:
>>>>>>>>> 
>>>>>>>>> Chris,
>>>>>>>>> 
>>>>>>>>> That sounds good to me.
>>>>>>>>> 
>>>>>>>>> I already have started on implementation, just to get familiar with
>>>>>>>>> the
>>>>>>>>> codebase and GFAPI.
>>>>>>>>> 
>>>>>>>>> Is there a public repo that we can use for collaboration?
>>>>>>>>> 
>>>>>>>>> Craig
>>>>>>>>> 
>>>>>>>>> On Jun 12, 2015, at 10:46 AM, chris holcombe <
>>>>>>>>>> chris.holcombe at canonical.com> wrote:
>>>>>>>>>> 
>>>>>>>>>> Craig,
>>>>>>>>>> 
>>>>>>>>>> I was actually planning on building the same tool set. I would like
>>>>>>>>>> to
>>>>>>>>>> work with you also on this if that's ok.
>>>>>>>>>> 
>>>>>>>>>> -Chris
>>>>>>>>>> 
>>>>>>>>>>> On 06/12/2015 10:43 AM, Jeff Darcy wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Hi everyone,
>>>>>>>>>>>> This summer I am an intern at Facebook working on the Gluster
>>>>>>>>>>>> team.
>>>>>>>>>>>> Part of
>>>>>>>>>>>> my project for the summer includes developing a set of coreutils
>>>>>>>>>>>> that
>>>>>>>>>>>> utilizes the Gluster C API natively.
>>>>>>>>>>>> 
>>>>>>>>>>>> This project is similar in nature to the NFS coreutils that some
>>>>>>>>>>>> of
>>>>>>>>>>>> you may
>>>>>>>>>>>> have heard about from the other Facebook engineers at the Gluster
>>>>>>>>>>>> summit
>>>>>>>>>>>> recently. I just wanted to reach out to the Gluster community to
>>>>>>>>>>>> gather
>>>>>>>>>>>> ideas, potential features, feedback, and direction.
>>>>>>>>>>>> 
>>>>>>>>>>>> The initial set of utilities that I am developing includes the
>>>>>>>>>>>> following:
>>>>>>>>>>>> 
>>>>>>>>>>>> * cat
>>>>>>>>>>>> * mkdir
>>>>>>>>>>>> * put (read from stdin and write to a file)
>>>>>>>>>>>> * mv
>>>>>>>>>>>> * ls
>>>>>>>>>>>> * rm
>>>>>>>>>>>> * tail
>>>>>>>>>>>> 
>>>>>>>>>>>> Again, any feedback will be welcome.
>>>>>>>>>>> Hi, Craig, and welcome to the project.  :)
>>>>>>>>>>> 
>>>>>>>>>>> There seems to be some overlap with a proposal Ragahavendra Talur
>>>>>>>>>>> sent
>>>>>>>>>>> out
>>>>>>>>>>> a couple of days ago.
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=https://docs.google.com/document/d/1yuRLRbdccx_0V0UDAxqWbz4g983q5inuINHgM1YO040/edit?usp%3Dsharing&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0A&s=8d55bb5770b8ed1d683a6908a05af32b79289735c537c660252fcaa7c690e162
>>>>>>>>>>> 
>>>>>>>>>>> This seems like an excellent opportunity to collaborate.  Ideally,
>>>>>>>>>>> I
>>>>>>>>>>> think
>>>>>>>>>>> it would be useful to have both an FTP-client-like "shell" and a
>>>>>>>>>>> set of
>>>>>>>>>>> standalone "one shot" commands, based on as much common code as
>>>>>>>>>>> possible.
>>>>>>>>>>> 
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> Gluster-devel mailing list
>>>>>>>>>>> Gluster-devel at gluster.org
>>>>>>>>>>> 
>>>>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=86la5Xg7nlxAzIR6E5c2v2SgQSd6VssYzB%2BklM3wf%2BI%3D%0A&s=28546cdc6fdf6f75f4cfa4b8260abc595eee96601a5f849ebb230ddbd1faf8b3
>>>>>>>> _______________________________________________
>>>>>>>> Gluster-devel mailing list
>>>>>>>> Gluster-devel at gluster.org
>>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>>>> _______________________________________________
>>>>>>> Gluster-devel mailing list
>>>>>>> Gluster-devel at gluster.org
>>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>>> _______________________________________________
>>>>>> Gluster-devel mailing list
>>>>>> Gluster-devel at gluster.org
>>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>>>> 
>>>>> _______________________________________________
>>>>> Gluster-devel mailing list
>>>>> Gluster-devel at gluster.org
>>>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>>> 
>>> _______________________________________________
>>> Gluster-devel mailing list
>>> Gluster-devel at gluster.org
>>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=vXGpsdu0VQkkhe9HG68hcfca%2F938GTXmmbz4t56hNMg%3D%0A&s=33c038573fdbc5f7914eefe2f0a5e8cfc57ea8c652a1cb42358e6d1a401628f1
>> _______________________________________________
>> Gluster-devel mailing list
>> Gluster-devel at gluster.org
>> https://urldefense.proofpoint.com/v1/url?u=http://www.gluster.org/mailman/listinfo/gluster-devel&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=ThH6JMKaB%2Fxgkh9d2jPjehcdps8B69L0q04jdBbZvX4%3D%0A&m=TAgCUd2QFR665sNN6ukXRFvoAv6uRiOVHk%2Bfk5BIhO8%3D%0A&s=8a2325c4cc557cc546ce9432f9f7055efa6f6af5d351046a3f230d63aa98a7c0
>> 