[Gluster-users] Shared VM disk/image on gluster for redundancy?

[Emmanuel Noobadmin]

On 6/29/10, Jeff Darcy <jdarcy at redhat.com> wrote:

> Not only would this work, but my impression is that it's a pretty common
> use of GlusterFS.  The one thing I'd add is that, if you already have or
> ever might have more than two application servers, you use cluster/nufa
> as well as cluster/replicate (which is what volgen's "RAID 1" actually
> does).  The basic idea here is to set up two (or more) subvolumes on
> each server, like this:
>
> 	srv0vol0	srv1vol0	srv2vol0
> 	srv0vol1	srv1vol1	srv2vol1
>
> Then you replicate "diagonally" with read-subvolume pointing to the top row:
>
> 	volume rep0
> 		type cluster/replicate
> 		option read-subvolume srv0vol0
> 		subvolumes srv0vol0 srv1vol1
> 	end-volume
>
> 	volume rep1
> 		type cluster/replicate
> 		option read-subvolume srv1vol0
> 		subvolumes srv1vol0 srv2vol1
> 	end-volume
>
> 	volume rep2
> 		type cluster/replicate
> 		option read-subvolume srv2vol0
> 		subvolumes srv2vol0 srv0vol1
> 	end-volume
>
> Lastly, you apply NUFA with "local-volume-name" on each node pointing to
> the replicated volume with its read-subvolume on the same machine.  So,
> on node 1:
>
> 	volume my_nufa
> 		type cluster/nufa
> 		option local-volume-name rep1
> 		subvolumes rep0 rep1 rep2
> 	end-volume
>
> With this type of configuration, files created on node 1 will be written
> to srv1vol0/srv2vol1 and read from srv1vol0.  Note that you don't need
> separate disks or anything to set up multiple volumes on a node; they
> can just be different directories, though if they're directories within
> the same local filesystem then "df" on the GlusterFS filesystem can be
> misleading.  Extending the approach from three servers to any N should
> be pretty obvious, and you can do the same thing with cluster/distribute
> instead of cluster/nufa (they actually use the same code) if strong
> locality is not a requirement.

Thank you very very much for the detailed explanation! Finally seeing
a definitive answer is really like light at the end of a long and
sleepless tunnel :)

> You're not really getting rid of heartbeat/failover delay, so much as
> relying on functionally equivalent behavior within GlusterFS.  Also,
> you'll still need some sort of heartbeat to detect that an application
> server has died.  Putting your images on GlusterFS makes it possible for
> guests on multiple machines to access them, but it's still a bad idea
> for them to do so simultaneously.

Switching image be manually done so it's not likely there will be
simultaneous access. Unless I can figure out if KVM has similar live
migration functionality as Xen's Remus.

So most likely I would run two or more physical machines with VM to
failover to each other to catch situations of a single machine
failure. Along with that a pair of storage server. In the case of a
total failure where both the primary & secondary VM dies physically,
roll in a new machine to load up the VM images still safe on the
gluster data servers.

So in this case would I be correct that my configuration, assuming a
basic 2 physical VM host server and 2 storage server would probably
look something like

volume rep0
	type cluster/replicate
	option read-subvolume vmsrv0vol0
	subvolumes vmsrv0vol0 datasrv0vol0 datasrv1vol0
end-volume


volume rep1
	type cluster/replicate
	option read-subvolume vmsrv1vol0
	subvolumes vmsrv1vol0 datasrv0vol0 datasrv1vol0
end-volume

volume my_nufa
	type cluster/nufa
	option local-volume-name rep0
	subvolumes rep0 rep1
end-volume

Or did I lose my way somewhere? :)
Does it make any sense to replicate across all 3 or should I simply
spec the VM servers with tiny drives and put everything on the gluster
storage which I suppose would impact performance severely?