[Gluster-devel] pluggability of some aspects in afr/nsr/ec

[Jeff Darcy]

> >> I want to understand if there is a possibility of exposing these as
> >> different modules that we can mix and match, using options.

It’s not only possible, but it’s easier than you might think.  If an
option is set (cluster.nsr IIRC) then we replace cluster/afr with
cluster/nsr-client and then add some translators to the server-side
stack.  A year ago that was just one nsr-server translator.  The
journaling part has already been split out, and I plan to do the same
with the leader-election parts (making them usable for server-side AFR
or EC) as well.  It shouldn’t be hard to control the addition and
removal of these and related translators (e.g. index) with multiple
options instead of just one.  The biggest stumbling block I’ve actually
hit when trying to do this with AFR on the server side is the *tests*,
many of which can’t handle delays on the client side while the server
side elects leaders and cross-connects peers.  That’s all solvable.  It
just would have taken more time than I had available for the experiment.

> precisely. I think switching is not that difficult once we make sure
> healing is complete. Switching is a rare operation IMO so we can
> probably ask the users to do stop/choose-new-value/start the volume
> after choosing the options. This way is simpler than to migrate
> between the volumes where you have to probably copy the data.

The two sets of metadata are *entirely* disjoint, which puts us in a
good position compared e.g. to DHT/tiering which had overlaps.  As long
as the bricks are “clean” switching back and forth should be simple.  In
fact I expect to do this a lot when we get to characterizing performance
etc.

> >> choose 1b, 2b it becomes nsr and 1a, 2a becomes afr/ec. In future
> >> if we come up with better metadata journals/stores it should be
> >> easy to plug them is what I'm thinking. The idea I have is based on
> >> the workload, users should be able to decide which pair of
> >> synchronization/metadata works best for them (Or we can also
> >> recommend based on our tests). Wanted to seek your inputs.

Absolutely.  As I’m sure you’re tired of hearing, I believe NSR will
outperform AFR by a significant margin for most workloads and
configurations.  I wouldn’t be the project’s initiator/leader if I
didn’t believe that, but I’m OK if others disagree.  We’ll find out
eventually.  ;)  More importantly, “most” is still not “all”.  Even by
my own reckoning, there are cases in which AFR will perform better or be
preferable for other reasons.  EC’s durability and space-efficiency
advantages make an even stronger case for preserving both kinds of data
paths and metadata arrangements.  That’s precisely why I want to make
the journaling and leader-election parts more generic.