[Gluster-users] # of replica != number pf bricks?

[Jeff Darcy]

> What happens if I have 3 peers for quorum. I create 3 bricks and want to have
> only two replicas in my volume.

The number of *bricks* must be a multiple of the replica count, but
quorum is based on the number of *servers* and there can be multiple
bricks per server.  Therefore, if you have servers A, B, and C with two
bricks each, you can do this:

   volume create foo replica 2 \
          A:/brick0 B:/brick0 C:/brick0 A:/brick1 B:/brick1 C:/brick1

First we'll combine this into the following two-way replica sets:

   A:/brick0 and B:/brick0
   C:/brick0 and A:/brick1
   B:/brick1 and C:/brick1

Then we'll distribute files among those three sets.  If one server fails
then we'll still have quorum (2/3) and each replica set will have at
least one surviving replica.  If two fail then neither of those things
will be true and we'll disable the volume.

In 4.0 we plan to improve on this by splitting bricks and creating the
necessary replica sets from the pieces ourselves.  Besides making
configuration simpler, this should remove the restriction on the number
of bricks being a multiple of the replica count, and also redistribute
load more evenly during or after a failure.  4.0 is a long way off,
though, so I probably shouldn't even be talking about it.  ;)