[Gluster-devel] RFC: Gluster.Next: Where and how DHT2 work/code would be hosted

[Shyam]

On 10/29/2015 07:29 PM, Jeff Darcy wrote:
> On October 29, 2015 at 9:12:46 AM, Shyam (srangana at redhat.com) wrote:
>> Will code that NSR puts up in master be ready to ship when 3.8 is
>> branched?
>
> Do we know when 3.8 will be branched?

This is an example not absolute, what I mean is when the next release is 
made.

>
>> I ask the above, as I think we need a *process*, and not an open ended
>> "put it where you want option", for the following reasons, - Something
>> that ships experimental is not to be consumed in production, so till
>> the point an/any effort is ready it can be housed in experimental - It
>> is not about how much of infra code is impacted, or how much code
>> would change, it is about readiness of the functionality
>
> I disagree.  From an operational perspective, unavoidable impact (not
> just on infrastructure but on any existing functionality) is *the* thing
> we want to avoid.  Experimental code that sits on disk, but isn’t even
> loaded into memory without an explicit request to enable it, carries
> little risk.  Experimental code that’s loaded and executed every time
> Gluster is run carries more risk, even if it’s just one line.

I assume this is about infra changes (as the first 2 points are for some 
reason squashed in my reader). I think what you state is infra (or other 
non-experimental) code impact due to changes by experimental/inprogress 
code, should be dealt with clearly and carefully so as to not impact 
regular functionality. In which case I *agree* and do not mean otherwise.

I think this sort of goes back to what Niels commented on my squashing a 
.patch and proposed using #define EXPERIMENTAL in this thread (or such 
methods).

>
>> - The intention is also to keep such WIP xlators segregated in master
>> - It is easy to control the "don't build/ship" directive for
>> everything under experimental, rather than make these decisions at a
>> per directory/xlator/module level
>
> They’re likely to make those decisions at the finer granularity anyway.
> Most people will probably only want to try out one experimental
> translator at a time.  Making them edit two makefiles (to enable
> “experimental” and then a specific translator within it) instead of just
> one doesn’t seem to get us very far.  Either way they’ll have to edit
> the specfile, and they’ll see a list of all the experimental translators
> they could enable.
>
>> - It is a clear message for anyone picking up these pieces to deploy
>> and play with etc.
>
> If having to edit makefiles and specfiles by hand isn’t enough, there
> are other things we could do to send such a clear message.  For example,
> we could require a special configure flag or glusterd startup option to
> enable management support for experimental features - not just whole
> translators but options within existing translators as well.
>
>> Coming to DHT2, irrespective of reason (1) above, all other reasons
>> for which NSR can stay outside of experimental holds good for DHT2 as
>> well.
>
> Perhaps.  I think that’s pretty clearly true for the DHT2 translator
> itself.  For the associated posix changes it’s less clearly true, but
> the modified version could go in storage/posix2 as easily as
> experimental/posix or experimental/posix2.

Yes, posix2 is where the new posix code would land, hence the comment on 
DHT2 being mostly similar in nature.

>
> IMO the main reason to have an “experimental” directory is one not
> mentioned yet - to put them in a separate RPM.  This is not the same as
> your “don’t build/ship” point above because they’ll get *built* anyway.

I think I should word my response more clearly, as "don't build/ship" is 
taken literally.

The choice of experimental as a separate entity, makes some of the 
choices presented below easier to implement/follow IMHO, which is what I 
am getting at.

Another thing I am getting at is, we *should* define a clear way to do 
such things, and not leave it open ended, which is where we seem to be 
headed below.

> They’ll just be separately installable - or not, for people who aren’t
> interested in experimental code.  Then we could combine that with the
> special glusterd startup option mentioned above to make the wole process
> of enabling or disabling experimental translators pretty seamless.
> Install the RPM, tweak the option, and you can use experimental code.
> Otherwise you can’t, and you get a nice clear error message if you try.
>
>
> So, here's what I think we should do (right now - subject to change).
>
>   1. We should create an "experimental" directory, and update makefiles
>      accordingly.

I am going to point back to the patch around this here, as it contains a 
README as well, which we can put these specifics into, 
http://review.gluster.org/#/c/12321/2

We can further that, or create a new one, I am fine either way.

>
>   2. The main Gluster 4.0 translators - dht2+posix, nsr* - should go into
>      the experimental directory and the specfile should put those
>      translators into a new RPM.
>
>   3. All experimental functionality, whether in its own translator or
>      otherwise, should be controlled by an option which is off by
>      default.

Ah! I think this is something akin to the "#define EXPERIMENTAL" 
suggestion and non-experimental code impact I guess, right?

>
>   4. Configuring an experimental feature should require a special
>      glusterd flag (plus installation of the experimental RPM).
>