[Gluster-Maintainers] [Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

Sun Oct 11 21:50:41 UTC 2015

On Fri, Oct 09, 2015 at 09:37:11AM +0530, Atin Mukherjee wrote:
> First of all my apologies for not going through the meeting blog before
> sending my thoughts on how we plan to maintain GlusterD 2.0 [1].

Because GlusterD-2.0 is quite a different approach, I am not sure it
should get included in the main glusterfs repository. At least for the
time being. If there is any tight integration, then I feel it should get
included, but if the new GlusterD 'only' prepares, starts and stops
processes, I think it can stay external.

I like a component based setup, where we can easily replace/update
single parts and are not required to update all running processes.

A patch to make GlusterD optional in the current repository should
probably be sent sooner than later. With such a patch, a pointer to
instructions that explain how to run with the new GlusterD would be a
requirement.

> This approach seems fine to me as long as we don't touch any existing
> xlators. How do we handle cases where other xlators get impacted by
> certain changes. Are we going to copy the whole translator in
> xlators/experimental and start working on it?

For major changes maybe. I prefer to see all the 'touching' done on the
normal xlators. configure.ac should set an EXPERIMENTAL (and maybe per
expiremental xlator?) #define in config.h. This #define can then be used
to #if/#else the 'touching' of the code. Using empty and non-empty
functions for these changes would be nice, that should keep the code
pretty readable, lots of #ifdef'ing in a function makes it difficult to
understand what the code is doing.

    #ifndef EXPERIMENTAL
    static void do_it (void) {}
    #else /* */
    static void do_it (void)
    {
        printf ("Hello Experiment!\n");
    }
    #endif

    void main (void)
    {
            do_it ();
    }

> Instead of all this wouldn't it be simpler to have development under a
> separate branch say "4.0-unstable" and we could disable CI on this
> branch till it becomes stable? Are we worried about pulling in the
> changes from this to master once the branch becomes stable?

This can be an option, but then I would rather see a branch per feature.
Whe na feature is ready, all its changes should get merged into the
master branch. I do not expect the different features to be ready at the
same time. Also regular re-basing of the feature branches on top of the
master branch would need to be done to prevent feature branches getting
out of sync.

This is a little similar to what linux-next does too. Many different
branches with new features and bug fixes get merged into a daily
linux-next. Merge conflicts get detected early, and people can test
linux-next and report problems that the features introduced.

HTH,
Niels

> 
> This is just my thought and I would like to get a clarity on this.
> 
> Thanks,
> Atin
> 
> [1] http://www.gluster.org/pipermail/gluster-devel/2015-October/046872.html
> 
> On 10/08/2015 11:35 PM, Shyam wrote:
> > Hi,
> > 
> > On checking yesterday's gluster meeting AIs and (later) reading the
> > minutes, for DHT2 here is what I gather and propose to do for $SUBJECT.
> > Feel free to add/negate any plans.
> > 
> > (This can also be discussed at [2])
> > 
> > -------------------------------------------------------------------
> > 1) Create a directory under the glusterfs master branch as follows,
> > ./xlators/*experimental*/dht2
> > ./xlators/*experimental*/posix2
> > 
> > See patch request at [2]
> > 
> > All code, design documents (work products in general) would go into this
> > directory.
> > 
> > 2) Code that compiles and does not cause CI failures could *potentially*
> > be merged with very few DHT2 dev folks assent.
> > 
> > There would possibly be no CI integration till we get something working,
> > so merges would be based on compile passing initially. Soon there would
> > be an attempt at getting unit testing integrated, so that code being
> > submitted is not abysmally horrendous
> > 
> > 3) Common framework code changes (if any) would be presented as a
> > separate commit request
> > 
> > 4) (Big problem) DHT2 requires glusterd changes to create a volume as
> > DHT2 and not DHT, this would be maintained as a .patch in the dht2
> > directory as above. This is so that people can play with DHT2 volumes if
> > interested. Integration of this piece either comes with glusterd 2.0 or
> > based on time lines of other events, in the current version of glusterd.
> > (if you are interested in seeing the current version of this patch, go
> > here [1])
> > -------------------------------------------------------------------
> > 
> > If there is some key disagreement on certain points like (2) above, then
> > we would need to bring in DHT2 code in parts so that it makes sense.
> > This is fine too, just that we would have 2 repos till we reach a point
> > of maturity in development.
> > 
> > -------------------------------------------------------------------
> > *Some issues with the approach:*
> > A) We need to ensure we do not ship xlators compiled from the
> > experimental directory
> > 
> > B) We need to possibly add a buddy maintainer for experimental
> > translator owners, who can help with the process of merging their changes.
> > 
> > C) I am not sure how this helps the review process, as initially xlator
> > development can be iffy and so we do not expect reviews to be stringent.
> > Later when we want to move this out of the experimental category, how do
> > we review the same now, and what actions do we take to ensure quality?
> > Won't we have the same bulk code review issue?
> > -------------------------------------------------------------------
> > 
> > Shameless plug: For quality and if an xlator plays well with other parts
> > of gluster the distaf framework of testing against possible graphs and
> > access protocols can be of immense help.
> > 
> > Shyam
> > 
> > [1]
> > https://github.com/ShyamsundarR/glusterfs/commit/663eeb98f6a51384c8745b8882e7c6c4f7b58a7c
> > 
> > [2] http://review.gluster.org/#/c/12321/1
> > _______________________________________________
> > Gluster-devel mailing list
> > Gluster-devel at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-devel
> _______________________________________________
> maintainers mailing list
> maintainers at gluster.org
> http://www.gluster.org/mailman/listinfo/maintainers
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/maintainers/attachments/20151011/9b7324da/attachment.sig>