[Gluster-devel] Gluster Coreutils

[Jeff Darcy]

> I'm Craig's manager for the duration of his internship @ FB, so I thought I'd
> better chime in here :).  As Craig mentioned, our project plan is to
> implement C-based CLI utilities similar to what we have for NFS CLI
> utilities (we'll be open sourcing this in the coming days so you can see
> what we have there).  Our time to completion is very aggressive for a
> project coded in C (~10 weeks or less), so we need to keep our project goals
> very focused (feature creep, architecture rabbit holes etc) to keep it on
> schedule and come out with a working "product" that scales to our use-cases.
> 
> As such, rather than have "too many cook in the kitchen", I'd propose the
> initial implementation of ls/tail/cat/put/mkdir/rm/stat/cp be implemented by
> Craig, and we'll put the code up on the public repo as early as possible for
> public review and comment.  We've definitely heard some great ideas from the
> list, and we've changed our approach to include:
> 
> 1. Shell based interaction
> 2. Single binary (behavior changes based on symlink; any cons to this?)
> 3. Separate code repo/packaging (we'll probably go C99 because of this since
> we are no longer bound by the legacy C89 standards of the GFS repo).
> 
> On the language debate (Python vs Go vs C), rather than get into this debate
> (which has good arguments on all sides) I'd simply encourage anyone who
> feels strongly about implementing the CLI in a different language to simply
> do so!  More choice and competition among tools is never a bad thing, as it
> will only make them both better.  Suffice it to say though, we feel strongly
> about the C based implementation so we are taking it on to provide parity
> with our NFS CLI utilities (which I'm working to get open-sourced in the
> coming weeks).
> 
> And finally, with respect to scaling: we feel reasonably confident we can
> scale the tools without any sort of proxy to work on the order of 100's of
> simultaneous CLI clients hitting a cluster.  We are going to experiment with
> some "proxy" approaches others to try to scale this to 1000's of CLI clients
> into a single cluster.  Though, the proxy will be optional as we feel the
> simplicity/reliability of a standalone client (hitting a standard
> load-balanced or DNS RR end point) will work well for vast majority of users
> (envision a "-p <proxy>" option in the tools for those who need to really
> scale them).

Thanks for this.  I was getting tempted to jump in and address some of the
"feature creep" myself, but it's a lot better for you to have done it.  :)
The point about "just do it" and choice/competition is IMO particularly
important with this crowd.