[Gluster-devel] Glusterd: A New Hope

[Anand Babu Periasamy]

On Tue, Mar 26, 2013 at 5:59 AM, Jeff Darcy <jdarcy at redhat.com> wrote:
> On 03/25/2013 11:59 PM, Anand Babu Periasamy wrote:
>> gluster meta-volume + zeromq for notification (pub/sub) will solve our
>> problems largely and still be light weight.  In a large scale
>> deployment, it is not a good idea to declare all the servers as
>> coordination servers. Since meta-volume is a regular distributed
>> replicated gluster volume, it can always be expanded later depending
>> on the load and availability requirements.
>
> How do you propose to solve the bootstrap problem?  How do we choose
> which subset of all servers would have pieces of this meta-volume?
> How do we coordinate its expansion, or detect and respond to failures of
> itscomponents?

Admin declares the config servers manually. Automatic option is a
disaster waiting to happen.

We should set the minimum number of config servers required by default
to 3, so that we can enforce quorum. Any failure would require the
admin to declare replacement config servers manually.

To declare config server, we would introduce a new gluster command.

Admin declares config servers right after peer probe and before
proceeding with volume creation.

Since most users start with 2 nodes for testing, we may add a --force
or --demo option.

I haven't slept all night. I am may be completely out of my mind :).

> What issues does 0MQ introduce with respect to daemons
> or network capabilities?

Yes we will have surprises for sure. For example, 0MQ is not thread
safe, short messages are buffered and there is no flush api. We have
to evaluate and see if it makes sense for our requirements.

I certainly do not like AMQP based implementations which are far more
complex to setup and manage than GlusterFS itself.

> It would be nice to have two or more proposals
> that have been been thought through to approximately the same level of
> detail (which isn't all that high really).  Otherwise we can't really
> make rational comparisons and choices.  Every approach to any
> non-trivial problem has its pitfalls.  If we don't look for them we risk
> choosing an approach with problems which are merely less obvious but no
> less severe.

I completely agree.
-ab



-- 
-ab

Imagination is more important than knowledge --Albert Einstein