[Gluster-users] how well will this work

[Brian Candler]

On Wed, Dec 26, 2012 at 11:24:25PM -0500, Miles Fidelman wrote:
> I find myself trying to expand a 2-node high-availability cluster
> from to a 4-node cluster.  I'm running Xen virtualization, and
> currently using DRBD to mirror data, and pacemaker to failover
> cleanly.

Not answering your question directly, but have you looked at Ganeti? This is
a front-end to Xen+LVM+DRBD (open source, written by Google) which makes it
easy to manage such a cluster, assuming DRBD is meeting your needs well at
the moment.

With Ganeti each VM image is its own logical volume, with its own DRBD
instance sitting on top, so you can have different VMs mirrored between
different pairs of machines.  You can migrate storage, albeit slowly (e.g. 
starting with A mirrored to B, you can break the mirroring then re-mirror A
to C, and then mirror C to D). Ganeti automates all this for you.

Another option to look at is Sheepdog, which is a clustered block-storage
device, but this would require you to switch from Xen to KVM.

> and performance seemed to be a
> problem in the configuration I'm thinking of.

With KVM at least, last time I tried performance was still very poor when
a VM image was being written to a file over gluster - I measured about
6MB/s.

However remember that each VM can directly mount glusterfs volumes
internally, and the performance of this is fine - and it also means you can
share data between the VMs.  So with some rearchitecture of your application
you may get sufficient performance for your needs.

Regards,

Brian.