[Gluster-devel] ZkFarmer

[Anand Babu Periasamy]

On Tue, May 8, 2012 at 4:08 PM, Ian Latter <ian.latter at midnightcode.org> wrote:
>> On 05/08/2012 12:27 AM, Ian Latter wrote:
>> > The equivalent configuration in a glusterd world (from
>> > my experiments) pushed all of the distribute knowledge
>> > out to the client and I haven't had a response as to how
>> > to add a replicate on distributed volumes in this model,
>> > so I've lost replicate.
>>
>> This doesn't seem to be a problem with replicate-first vs.
> distribute-first,
>> but with client-side vs. server-side deployment of those
> translators.  You
>> *can* construct your own volfiles that do these things on
> the servers.  It will
>> work, but you won't get a lot of support for it.  The
> issue here is that we
>> have only a finite number of developers, and a
> near-infinite number of
>> configurations.  We can't properly qualify everything.
> One way we've tried to
>> limit that space is by preferring distribute over
> replicate, because replicate
>> does a better job of shielding distribute from brick
> failures than vice versa.
>> Another is to deploy both on the clients, following the
> scalability rule of
>> pushing effort to the most numerous components.  The code
> can support other
>> arrangements, but the people might not.
>
> Sure, I have my own vol files that do (did) what I wanted
> and I was supporting myself (and users); the question
> (and the point) is what is the GlusterFS *intent*?  I'll
> write an rsyncd wrapper myself, to run on top of Gluster,
> if the intent is not allow the configuration I'm after
> (arbitrary number of disks in one multi-host environment
> replicated to an arbitrary number of disks in another
> multi-host environment, where ideally each environment
> need not sum to the same data capacity, presented in a
> single contiguous consumable storage layer to an
> arbitrary number of unintelligent clients, that is as fault
> tolerant as I choose it to be including the ability to add
> and offline/online and remove storage as I so choose) ..
> or switch out the whole solution if Gluster is heading
> away from my  needs.  I just need to know what the
> direction is .. I may even be able to help get you there if
> you tell me :)

Rsync'ing the vol spec files is the simplest and elegant approach. It
is how glusterfs originally handled config files. How ever elastic
volume management (online volume management operations) requires
synchronized online changes to volume spec files. This requires
GlusterFS to manage volume specification files internally. That is why
we brought glusterd in 3.1.

Real question is: do we want to keep the volume spec files on all
nodes (fully distributed) or few selected nodes.

>
>> BTW, a similar concern exists with respect to replication
> (i.e. AFR) across
>> data centers.  Performance is going to be bad, and there's
> not going to be much
>> we can do about it.
>
> Hmm .. that depends .. these sorts of statements need
> context/qualification (in bandwidth and latency terms).  For
> example the last multi-site environment that I did
> architecture for was two DCs set 32kms apart with a
> redundant 20Gbps layer-2 (ethernet) stretch between
> them - latency was 1ms average, 2ms max (the fiber
> actually took a 70km path).  Didn't run Gluster on it, but
> we did stretch a number things that "couldn't" be stretched.
>
>
>> > But in this world, the client must
>> > know about everything and the server is simply a set
>> > of served/presented disks (as volumes).  In this
>> > glusterd world, then, why does any server need to
>> > know of any other server, if the clients are doing all of
>> > the heavy lifting?
>>
>> First, because config changes have to apply across
> servers.  Second, because
>> server machines often spin up client processes for things
> like repair or
>> rebalance.
>
> Yep, but my reading is that the config's that the servers
> need are local - to make a disk a share (volume), and
> that as you've described the rest are "client processes"
> (even when on something built as a "server"), so if you
> catered for all clients then you'd be set?  I.e. AFR now
> runs in the client?
>
>
> And I am sick of the word-wrap on this client .. I think
> you've finally convinced me to fix it ... what's normal
> these days - still 80 chars?

I used to line-wrap (gnus and cool emacs extensions). It doesn't make
sense to line wrap any more. Let the email client handle it depending
on the screen size of the device (mobile / tablet / desktop).

-- 
Anand Babu Periasamy
Blog [http://www.unlocksmith.org]

Imagination is more important than knowledge --Albert Einstein