[Gluster-users] Performance issues on GlusterFS with KVM/qcow2

[Tobias Andersson]

Hi,

I am currently doing some performance tests with GlusterFS to see if it 
would be possible to replace our current storage infrastructure with 
SATA/RAID10 servers in a GlusterFS replica. Everything looks good on the 
paper and the setup was smooth.

However when it comes to write speed its not really what I was hoping 
for. Here are some stats:

---------

Write test 1, VM-HOST (Debian 6, GlusterFS on /vm_storage):

storage1:/cloud on /vm_storage type fuse.glusterfs 
(rw,allow_other,default_permissions,max_read=131072)

echo 3 > /proc/sys/vm/drop_caches
time dd if=/dev/zero of=./bigfile bs=1M count=5000

Result:

5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 101.464 s, 51.7 MB/s

real    1m41.725s
user    0m0.004s
sys     0m3.104s

---------

Write test 2, VM (Debian 6, VirtIO, stored as qcow2 on /vm_storage):

/dev/vda2 on / type ext4 (rw,errors=remount-ro)

echo 3 > /proc/sys/vm/drop_caches
time dd if=/dev/zero of=./bigfile bs=1M count=5000

Result:

5000+0 records in
5000+0 records out
5242880000 bytes (5.2 GB) copied, 796.309 s, 6.6 MB/s

real    13m16.626s
user    0m0.000s
sys     0m3.700s

---------

Conclusion:

51 MB/s on KVM-HOST is as expected, my uplink goes almost full with 
around 400-450Mbps to each GlusterFS storage node.

The problem appears only when doing benchmark within the KVM Virtual 
Machine. Write speeds are really poor (6.6 MB/s), however read is fine.

I've tried both 3.2.5 and 3.3 beta2 with no luck.

Is this a known issue? Have I missed something? Some performance tweaks 
maybe?

Best regards,

-- 
Tobias Andersson
------
Please avoid sending me Word, Excel or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html