[Gluster-users] Slow speed with Gluster Backed Xen DomU's
samppah at neutraali.net
Tue Jul 20 21:39:51 UTC 2010
Sorry for the delay with my answer. I got your message earlier but I was
hoping that someone else would answer with more precise information.
I did some testing almost a year ago to use GlusterFS to store Xen images.
I was testing with pre 2.10 version of GlusterFS but my findings were very
similar to yours. Writing speed was fine when done from Dom0 to GlusterFS
mount, I was even able to max out GigE link with my cheap hardware.
However when I did same tests on DomU (image inside the GlusterFS mount)
the write speeds dropped drastically. Read speeds were ok all the time.
Unfortunately I didn't have time to investigate it much further back then.
Although I did some performance tests with Bonnie++ on Dom0 and noticed
that block write speeds were almost the same as write speeds in general on
DomU. So, my guess is that block write speeds needs somekind of on
> Hope your day has been going well. I tried emailing this in a week or
> so ago, but i do not think that it got emailed to the group. I am just
> trying again, but apologies in advanced if you have already read this.
> I am currently using GlusterFS as a distributed SAN backend for the
> Xen based cloud platform we are developing.
> We deploy Xen virtuals on pairs of servers using GlusterFs V3 in
> replicate mode on Debian Stable (Lenny) with Xen 3.2.1 as the
> hypervisor. I am currently experiencing a weird issue where within the
> virtual machines (DomU) running on a GlusterFS mount only receive
> around 10-18MB/s write speeds, but full speed reads.
> Our hardware for each node is Dual Core Xeon Processors, 8GB of RAM
> and 4 * High Speed SATA drives (RAID 10, around 160MB/s writes and
> If I write a file to the Gluster mount in the Dom0 (host) we receive
> around 90-100MB/s writes (maxing out the GigE link). If I run the
> virtual machine on the disks without Gluster I get much higher speeds
> within the DomU of around 80-90MB/s.
> This slow down only appears to occur on writes. Does anyone with a
> better understanding of GlusterFS, Fuse and filesystems have an idea
> why this is slowing down. The underlying file system is Ext3 using
> TAP:AIO within Xen to connect to a file image based disk. This is
> without using gluster fuse client (what benefits does this give?) and
> Gluster version 3.0.4.
More information about the Gluster-users