[Gluster-devel] Infiniband slow throughput

[nathan at robotics.net]

Local disks on each box look ok:

[root at cluster0 export]# sync ; time (dd if=/dev/zero of=/home/export/foo.bar bs=4k count=512K ; sync )
524288+0 records in
524288+0 records out
2147483648 bytes (2.1 GB) copied, 17.0863 seconds, 126 MB/s

real    0m27.223s
user    0m0.440s
sys     0m8.461s
[root at cluster0 export]#



Infiniband throughput IBoIP looks ok:

[root at cluster0 src]# iperf -c 172.16.0.201
------------------------------------------------------------
Client connecting to 172.16.0.201, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 172.16.0.200 port 36805 connected with 172.16.0.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.88 GBytes  1.62 Gbits/sec



Gluster:

[root at cluster0 src]# glusterfs -f /usr/local/etc/glusterfs/client.vol -n brick1 /share
[root at cluster0 share]# sync ; time (dd if=/dev/zero of=/share/foo.bar 
bs=4k count=512K ; sync )
524288+0 records in
524288+0 records out
2147483648 bytes (2.1 GB) copied, 222.344 seconds, 9.7 MB/s

real    3m42.384s
user    0m0.168s
sys     0m1.016s



My configs:

http://share.robotics.net/client.vol
http://share.robotics.net/server.vol



><>
Nathan Stratton                                CTO, BlinkMind, Inc.
nathan at robotics.net                         nathan at blinkmind.com
http://www.robotics.net                        http://www.blinkmind.com