[Gluster-users] how well will this work

[Miles Fidelman]

Brian Candler wrote:
> On Wed, Dec 26, 2012 at 11:24:25PM -0500, Miles Fidelman wrote:
>> I find myself trying to expand a 2-node high-availability cluster
>> from to a 4-node cluster.  I'm running Xen virtualization, and
>> currently using DRBD to mirror data, and pacemaker to failover
>> cleanly.
> Not answering your question directly, but have you looked at Ganeti? This is
> a front-end to Xen+LVM+DRBD (open source, written by Google) which makes it
> easy to manage such a cluster, assuming DRBD is meeting your needs well at
> the moment.

I keep looking at Ganeti, played with it a bit in a test installation.  
It does a lot, but it falls short in two regards:

- it doesn't really have an auto-failover function (it keeps getting 
closer, but no cigar, at least last time I looked) - you either need 
intervene manually on a node failure, or you need to add something like 
pacemaker, and the plumbing starts to get very confused

- the second, you've identified
> With Ganeti each VM image is its own logical volume, with its own DRBD
> instance sitting on top, so you can have different VMs mirrored between
> different pairs of machines.  You can migrate storage, albeit slowly (e.g.
> starting with A mirrored to B, you can break the mirroring then re-mirror A
> to C, and then mirror C to D). Ganeti automates all this for you.

This is precisely what I'm hoping to get past with a cluster file-system.
> Another option to look at is Sheepdog, which is a clustered block-storage
> device, but this would require you to switch from Xen to KVM.

You nailed it.  Sheepdog is architected for nodes that combine storage 
and processing.  Sheepdog on Xen would be ideal.  Sigh....

> With KVM at least, last time I tried performance was still very poor when
> a VM image was being written to a file over gluster - I measured about
> 6MB/s.
>
> However remember that each VM can directly mount glusterfs volumes
> internally, and the performance of this is fine - and it also means you can
> share data between the VMs.  So with some rearchitecture of your application
> you may get sufficient performance for your needs.
>

Thanks!

Miles Fidelman

-- 
In theory, there is no difference between theory and practice.
In practice, there is.   .... Yogi Berra