[Gluster-users] Performance issues on GlusterFS with KVM/qcow2

[Andreas Kurz]

On 01/25/2012 01:35 PM, Tobias Andersson wrote:
> Hi,
> 
> I am currently doing some performance tests with GlusterFS to see if it
> would be possible to replace our current storage infrastructure with
> SATA/RAID10 servers in a GlusterFS replica. Everything looks good on the
> paper and the setup was smooth.
> 
> However when it comes to write speed its not really what I was hoping
> for. Here are some stats:
> 
> ---------
> 
> Write test 1, VM-HOST (Debian 6, GlusterFS on /vm_storage):
> 
> storage1:/cloud on /vm_storage type fuse.glusterfs
> (rw,allow_other,default_permissions,max_read=131072)
> 
> echo 3 > /proc/sys/vm/drop_caches
> time dd if=/dev/zero of=./bigfile bs=1M count=5000
> 
> Result:
> 
> 5000+0 records in
> 5000+0 records out
> 5242880000 bytes (5.2 GB) copied, 101.464 s, 51.7 MB/s
> 
> real    1m41.725s
> user    0m0.004s
> sys     0m3.104s
> 
> ---------
> 
> Write test 2, VM (Debian 6, VirtIO, stored as qcow2 on /vm_storage):

This might be interesting regarding qcow2 performance:

http://www.linux-kvm.org/page/Qcow2

Regards,
Andreas

-- 
Need help with High Availability clustering?
http://www.hastexo.com/now

> 
> /dev/vda2 on / type ext4 (rw,errors=remount-ro)
> 
> echo 3 > /proc/sys/vm/drop_caches
> time dd if=/dev/zero of=./bigfile bs=1M count=5000
> 
> Result:
> 
> 5000+0 records in
> 5000+0 records out
> 5242880000 bytes (5.2 GB) copied, 796.309 s, 6.6 MB/s
> 
> real    13m16.626s
> user    0m0.000s
> sys     0m3.700s
> 
> ---------
> 
> Conclusion:
> 
> 51 MB/s on KVM-HOST is as expected, my uplink goes almost full with
> around 400-450Mbps to each GlusterFS storage node.
> 
> The problem appears only when doing benchmark within the KVM Virtual
> Machine. Write speeds are really poor (6.6 MB/s), however read is fine.
> 
> I've tried both 3.2.5 and 3.3 beta2 with no luck.
> 
> Is this a known issue? Have I missed something? Some performance tweaks
> maybe?
> 
> Best regards,
> 



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 222 bytes
Desc: OpenPGP digital signature
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120125/db015522/attachment.sig>