[Gluster-users] question re. current state of art/practice

[Brian Candler]

On Wed, Feb 15, 2012 at 08:22:18PM -0500, Miles Fidelman wrote:
> We've been running a 2-node, high-availability cluster - basically xen
> w/ pacemaker and DRBD for replicating disks.  We recently purchased 2
> additional,  servers, and I'm thinking about combining all 4 machines
> into a 4-node cluster - which takes us out of DRBD space and requires
> some other kind of filesystem replication.
> 
> Gluster, Ceph, Sheepdog, and XtreemFS seem to keep coming up as things
> that might work, but... Sheepdog is too tied to KVM

... although if you're considering changing DRBD->Gluster, then changing
Xen->KVM is perhaps worth considering too?

Having VM images as single files in a filesystem arguably allows you to back
them up more easily, and if there are hotspots you can identify them or
shift load around manually.  But I do like the idea of the fully distributed
block device provided by sheepdog.

> i.  Is it now reasonable to consider running Gluster and Xen on the same
> boxes, without hitting too much of a performance penalty?

I have been testing Gluster on 24-disk nodes :
  - 2 HBAs per node (one 16-port and one 8-port)
  - single CPU chip (one node is dual-core i3, one is quad-core Xeon)
  - 8GB RAM
  - 10G ethernet
and however I hit it, the CPU is mostly idle. I think the issue for you is
more likely to be one of latency rather than throughput or CPU utilisation,
and if you have multiple VMs accessing the disk concurrently then latency
becomes less important.

However, I should add that I'm not running VMs on top of this, just doing
filesystem tests (and mostly reads at this stage).

For what gluster 3.3 will bring to the table, see this:
http://community.gluster.org/q/can-i-use-glusterfs-as-an-alternative-network-storage-backing-for-vm-hosting/

Regards,

Brian.