[Gluster-devel] ZkFarmer

[Ian Latter]

> On 05/08/2012 12:27 AM, Ian Latter wrote:
> > The equivalent configuration in a glusterd world (from
> > my experiments) pushed all of the distribute knowledge
> > out to the client and I haven't had a response as to how
> > to add a replicate on distributed volumes in this model,
> > so I've lost replicate.
> 
> This doesn't seem to be a problem with replicate-first vs.
distribute-first,
> but with client-side vs. server-side deployment of those
translators.  You
> *can* construct your own volfiles that do these things on
the servers.  It will
> work, but you won't get a lot of support for it.  The
issue here is that we
> have only a finite number of developers, and a
near-infinite number of
> configurations.  We can't properly qualify everything. 
One way we've tried to
> limit that space is by preferring distribute over
replicate, because replicate
> does a better job of shielding distribute from brick
failures than vice versa.
> Another is to deploy both on the clients, following the
scalability rule of
> pushing effort to the most numerous components.  The code
can support other
> arrangements, but the people might not.

Sure, I have my own vol files that do (did) what I wanted
and I was supporting myself (and users); the question 
(and the point) is what is the GlusterFS *intent*?  I'll
write an rsyncd wrapper myself, to run on top of Gluster,
if the intent is not allow the configuration I'm after 
(arbitrary number of disks in one multi-host environment 
replicated to an arbitrary number of disks in another 
multi-host environment, where ideally each environment 
need not sum to the same data capacity, presented in a
single contiguous consumable storage layer to an 
arbitrary number of unintelligent clients, that is as fault 
tolerant as I choose it to be including the ability to add 
and offline/online and remove storage as I so choose) .. 
or switch out the whole solution if Gluster is heading 
away from my  needs.  I just need to know what the 
direction is .. I may even be able to help get you there if
you tell me :)


> BTW, a similar concern exists with respect to replication
(i.e. AFR) across
> data centers.  Performance is going to be bad, and there's
not going to be much
> we can do about it.

Hmm .. that depends .. these sorts of statements need 
context/qualification (in bandwidth and latency terms).  For 
example the last multi-site environment that I did 
architecture for was two DCs set 32kms apart with a 
redundant 20Gbps layer-2 (ethernet) stretch between 
them - latency was 1ms average, 2ms max (the fiber 
actually took a 70km path).  Didn't run Gluster on it, but 
we did stretch a number things that "couldn't" be stretched.

 
> > But in this world, the client must
> > know about everything and the server is simply a set
> > of served/presented disks (as volumes).  In this
> > glusterd world, then, why does any server need to
> > know of any other server, if the clients are doing all of
> > the heavy lifting?
> 
> First, because config changes have to apply across
servers.  Second, because
> server machines often spin up client processes for things
like repair or
> rebalance.

Yep, but my reading is that the config's that the servers 
need are local - to make a disk a share (volume), and 
that as you've described the rest are "client processes" 
(even when on something built as a "server"), so if you 
catered for all clients then you'd be set?  I.e. AFR now 
runs in the client?


And I am sick of the word-wrap on this client .. I think
you've finally convinced me to fix it ... what's normal
these days - still 80 chars?




--
Ian Latter
Late night coder ..
http://midnightcode.org/