[Gluster-users] Gluster / KVM Filesystem Benchmarks

[Andrei Mikhailovsky]

Guys, do you know if this patch supports glusterfs over rdma? 

I am running glusterfs over infiniband and kvm performance is so nasty. As an example, I get around 600-800 mb/s for read/write on the glusterfs partition mounted on kvm server. However, vms stored on this partition can only read/write around 40-50mb/s. I would love to try this patch if there is rdma support. 

Thanks 





----- Original Message -----

From: "Joe Topjian" <joe at topjian.net> 
To: gluster-users at gluster.org 
Sent: Wednesday, 5 September, 2012 6:47:09 AM 
Subject: [Gluster-users] Gluster / KVM Filesystem Benchmarks 

Hello, 


I did a few filesystem benchmarks with Gluster (3.3) and KVM using iozone and have compiled a spreadsheet with the results: 


https://docs.google.com/open?id=0B6GzZufzohwFZmozTFRSSHk5T0E 


Just a heads up: It is an Excel spreadsheet. 


All of the details that were used to generate the results are described in the spreadsheet. Of most interest would be the second tab titled "Gluster". The results that do not have "vm" in the description were iozone procs running directly on a mounted replicated Gluster volume (2 bricks). The "vm" results are iozone procs running in KVM virtual machines stored in qcow2 files. 


The first tab, General, is just some simple non-Gluster benchmarks that I ran for comparison. 


The third tab, Gluster old, was me doing iozone benchmarks on files with sizes ranging from 8mb to 512mb. I noticed that there was very little difference in the results so I decided to work with only 128mb and 256mb sized files. 


If you do not have access to Excel or something compatible, you can still view most of the information in the Google Doc. Here is a jpeg image of the main graph that was generated: 


https://docs.google.com/open?id=0B6GzZufzohwFWGtFS3I5UEllTkU 


Questions I have: 


* The "optimized settings" that I used were pulled from a Gluster Performance Tuning presentation. It doesn't look like the settings did very much in terms of optimization. Can someone comment on these settings? Are there better settings to use? 


* I'm a bit confused at how the KVM / qcow2 reads are much higher than the reads directly on the Gluster volume. Any idea why that is? 


* I ran all tests with the cache-io translator on and off. Like the "optimized settings", it wasn't of much use. Did I use this incorrectly? 


* The reason I did all tests with 128mb and 256mb sized files was to highlight the very bizarre trait where certain increments (16, 64, 256) gave very poor results while increments such as 8, 32, and 128 had good results. Any idea why that is? 

* Can anyone comment on if these results are of any use? Or are the stats I collect and the way I collected them incorrect? 


Please let me know if anyone has any questions or needs anything clarified. 


Thanks, 
Joe 
_______________________________________________ 
Gluster-users mailing list 
Gluster-users at gluster.org 
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20120909/12e9f916/attachment.html>