[Gluster-devel] a union to two stripes to fourteen mirrors...
kbenson at a-1networks.com
Tue Nov 20 18:13:01 UTC 2007
Jerker Nyberg wrote:
> I'm trying out different configurations of GlusterFS. I have 7 nodes
> each with two 320 GB disks where 300 GB om each disk is for the
> distributed file system.
> Each node is called N. Every file system is on the server side mirrored
> to the other disk on the next node, wrapped around so that the last node
> mirrors its disk to the first. invented. The real config is included in
> the end of this mail.
> fs(1) = a file system on the first disk
> fs(2) = a file system on the second disk
> n(I, fs(J)) = the fs J on node I
> afr(N .. M) = mirror the volumes
> stripe(N .. M) = stripe the volumes
> Forw(N) = afr(fs(1), node(N+1, fs(2))
> Back(N) = afr(fs(2), node(N-1, fs(1))
> FStr(N .. M) = stripe(n(N, Forw(N)) .. n(N+i, Forw(N+1)) .. n(M, Forw(M)))
> BStr(N .. M) = stripe(n(N, Back(N)) .. n(N+i, Back(N+1)) .. n(M, Back(M)))
> mount /glusterfs = union(FStr(1 .. 7), BStr(1..7))
> The goal was to get good performance but also redundancy. But this setup
> will not will it? The stripes will not work when a part of is gone and
> the union will not not magically find the other part of a file on the
> other stripe? And where to put the union namespace for good performance?
> But my major question is this: I tried to stripe a single stripe (not
> using union on the client, just striping on the servers which in turn
> mirrored) When rsync'ing in data on it on a single server things worked
> fine, but when I put some load on it from the other nodes (dd'ing in and
> out some large files) the glusterfsd's on the first server died... Do
> you want me to check this up more and try to reproduce and narrow down
> the problem, or is this kind of setup in general not a good idea?
I don't think the developers would ever consider a situation where
glusterfsd dies acceptable, so I'm sure they would want info on why it dies.
I can't seem to correlate your spec files with the layout you gave a
above. My understanding of your spec files makes it look like this to me:
Forw(N) = afr(fs(1), node(N+1, fs(2))
FStr(N .. M) = stripe(n(1, Forw(1)) .. n(N, Forw(N)))
In either case you have a problem if you are looking for high
availability, as the loss of any single node will reduce the cluster to
an unusable state (unless striping does some stuff I don't know about).
Since each AFR is defined on the main node itself, the client doesn't
know it's actually putting files on node N and N+1, it's only talking to
N. If N dies, the client has no way of knowing it can still read and
write from the other AFR member, N+1. Moving the AFR config to the
client would fix this.
Maybe that's why you had the Back(N) stuff, but I'm not sure I
understand what you were trying to say with that... Were you trying to
define the same afr share through two different servers with two
different (but equivalent) configs?
More information about the Gluster-devel