[Gluster-devel] State of the 4.0 World

Sat Jun 11 05:05:21 UTC 2016

----- Original Message -----
> From: "Dan Lambright" <dlambrig at redhat.com>
> To: "Gluster Devel" <gluster-devel at gluster.org>
> Sent: Saturday, June 11, 2016 12:42:49 AM
> Subject: Re: [Gluster-devel] State of the 4.0 World
> 
> 
> 
> ----- Original Message -----
> > From: "Jeff Darcy" <jdarcy at redhat.com>
> > To: "Gluster Devel" <gluster-devel at gluster.org>
> > Sent: Tuesday, May 3, 2016 11:50:30 AM
> > Subject: [Gluster-devel] State of the 4.0 World
> > 
> > One of my recurring action items at community meetings is to report to
> > the list on how 4.0 is going.  So, here we go.
> > 
> > The executive summary is that 4.0 is on life support.  Many features
> > were proposed - some quite ambitious.  Many of those *never* had anyone
> > available to work on them.  Of those that did, many have either been
> > pulled forward into 3.8 (which is great) or lost what resources they had
> > (which is bad).  Downstream priorities have been the biggest cause of
> > those resource losses, though other factors such as attrition have also
> > played a part.  Net result is that, with the singular exception of
> > GlusterD 2.0, progress on 4.0 has all but stopped.  I'll provide more
> > details below.  Meanwhile, I'd like to issue a bit of a call to action
> > here, in two parts.
> > 
> >  * Many of the 4.0 sub-projects are still unstaffed.  Some of them are
> >    in areas of code where our combined expertise is thin.  For example,
> >    "glusterfsd" is where we need to make many brick- and
> >    daemon-management changes for 4.0, but it has no specific maintainer
> >    other than the project architects so nobody touches it.  Over the
> >    past year it has been touched by fewer than two patches per month,
> >    mostly side effects of patches which were primarily focused elsewhere
> >    (less than 400 lines changed).  It can be challenging to dive into
> >    such a "fallow" area, but it can also be an opportunity to make a big
> >    difference, show off one's skill, and not have to worry much about
> >    conflicts with other developers' changes.  Taking on projects like
> >    these is how people get from contributing to leading (FWIW it's how I
> >    did), so I encourage people to make the leap.
> > 
> >  * I've been told that some people have asked how 4.0 is going to affect
> >    existing components for which they are responsible.  Please note that
> >    only two components are being replaced - GlusterD and DHT.  The DHT2
> >    changes are going to affect storage/posix a lot, so that *might* be
> >    considered a third replacement.  JBR (formerly NSR) is *not* going to
> >    replace AFR or EC any time soon.  In fact, I'm making significant
> >    efforts to create common infrastructure that will also support
> >    running AFR/EC on the server side, with many potential benefits to
> >    them and their developers.  However, just about every other component
> >    is going to be affected to some degree, if only to use the 4.0
> >    CLI/volgen plugin interfaces instead of being hard-coded into their
> >    current equivalents.  4.0 tests are also expected to be based on
> >    Distaf rather than TAP (the .t infrastructure) so there's a lot of
> >    catch-up to be done there.  In other cases there are deeper issues to
> >    be resolved, and many of those discussions - e.g. regarding quota or
> >    georep - have already been ongoing.  There will eventually be a
> >    Gluster 4.0, even if it happens after I'm retired and looks nothing
> >    like what I describe below.  If you're responsible for any part of
> >    GlusterFS, you're also responsible for understanding how 4.0 will
> >    affect that part.
> > 
> > With all that said, I'm going to give item-by-item details of where we
> > stand.  I'll use
> > 
> > http://www.gluster.org/community/documentation/index.php/Planning40
> > 
> > as a starting point, even though (as you'll see) in some ways it's out
> > of date.
> > 
> > * GlusterD 2 is still making good progress, under Atin's and Kaushal's
> >    leadership.  There are designs for most of the important pieces, and
> >    a significant amount of code which we should be able to demo soon.
> > 
> >  * DHT2 had been making good progress for a while, but has been stalled
> >    recently as its lead developer (Shyam) has been unavailable.
> >    Hopefully we'll get him back soon, and progress will accelerate
> >    again.
> 
> DHT-2 will consolidate metadata on a server. This has the potential to help
> gluster's tiering implementation significantly, as it will not need to
> replicate directories on both the hot and cold tier. Chatting with Shyam,
> there appears to be three work items related to tiering and DHT-2.
> 
> 1.
> 
> An unmodified tiering translator "should" work with DHT-2. But to realize
> DHT-2's benefits, the tiering translator would need to be modified so
> metadata related FOPs are directed to only go to the tier on which the
> metadata resides.
> 
> 2.
> 
> "metadata" refers to directories, but (per my understanding), it could
> possibly include the file's inode as well. This is a choice- whether or not
> to include the inode in the metadata server is an technical investigation to
> undertake.
> 
> 3.
> 
> Tier's database is currently SQLite, but it has been understood from day one
> that we may wish to move to a different database or algorithm. RocksDB is
> one candidate that is an attractive alternative. It is used in Ceph and
> gluster-friendly Facebook, and we can leverage their expertise with it. If
> DHT-2 ends up using RocksDB this furthers the argument to switch to it. We
> would like to generalize the interface tier uses to the database (libgfdb).

I agree with Dan. A faster and scalable Heat Meta-Data store is always good, also a data store that
can give complex querying capabilities like SQLite. If these requirements addressed by the new Heat Meta-Data
Store then we have the holy grail of Heat Meta-Data store :).

> 
> 
> 
> > 
> >  * Sharding got pulled forward because of its importance for other
> >    efforts, so it's no longer a 4.0 feature.
> > 
> >  * Client-side caching has been dropped for now, though it could still
> >    return with a new design based on the lease infrastructure.
> > 
> >  * Data classification (beyond just tiering) has been dropped.
> > 
> >  * Multiple-network support and network QoS are very much still part of
> >    the 4.0 plan as far as I'm concerned, but there's still nobody
> >    available to work on them.
> > 
> >  * "Better brick management" is also still an un-resourced part of the
> >    4.0 plan.  A lot of the higher-level logic will go into Heketi, but
> >    exporting multiple bricks through a single daemon (and port) can't
> >    be.
> > 
> >  * Compression/dedup have been dropped.
> > 
> >  * Composite operations are already being implemented, either as part of
> >    3.8 or as part of the Samba/Ganesha efforts depending on how you look
> >    at it, so that's not a 4.0 feature any more.
> > 
> >  * Stat/xattr caching (on the server) is a bit of a question mark.  On
> >    the one hand, it should be pretty simple to implement.  On the other
> >    hand, nobody has made even a minimal effort to do so.  Recent events
> >    have also raised the issue of needing to do this for correctness
> >    (especially around maintaining ctime across replicas) as well as
> >    performance.  This would be a *great* opportunity for a currently
> >    junior/novice Gluster contributor to make their mark.
> > 
> >  * Code generation already exists, and is actively being used to
> >    implement other 4.0 features.  My only other comment here is that
> >    people should start using it instead of continuing to use macros in
> >    many cases.  Every macro we add is another little nugget of technical
> >    debt, causing all sorts of headaches for anyone who has to edit or
> >    debug the code later.  Please do your part to stamp out macro abuse.
> > 
> >  * Management plugins are part of the GlusterD 2 plan.
> > 
> >  * Performance monitoring etc. (last item on list) has been dropped, for
> >    lack of a well defined scope or requirements.
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> > 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 