[Gluster-devel] Infiniband slow throughput

[nathan at robotics.net]

On Wed, 10 Sep 2008, Anand Avati wrote:

> Which version of glusterfs are you using? Please load write-behind in the
> client volume file (over unify). Have you tried ib-verbs transport (instead
> of ib-sdp) ?

ib-verbs is faster, however I am not seeing anywhere near what I think I 
should be seeing. I could try write-behind, but I wanted to get stock up 
to some reasonable numbers first. I can write the file to local disk in 
28.818 seconds. I can understand infiniband being slower then local write, 
but not 5 times slower! Also I am not trying unify yet, just one client to 
one block.

I am running latest TLA with 2.7.3glfs10 fuse.

[root at cluster0 ~]# sync ; time (dd if=/dev/zero of=/share/foo.bar bs=4k 
count=512K ; sync )
524288+0 records in
524288+0 records out
2147483648 bytes (2.1 GB) copied, 158.164 seconds, 13.6 MB/s

real    2m39.045s
user    0m0.108s
sys     0m1.420s