[Gluster-users] RDMA Performance ?'s with 3.4.0

[Ryan Aydelott]

In some basic testing using 2 RDMA gluster servers configured as a replicated pair, I'm seeing better significantly better performance with IPoIB versus RDMA only volumes (almost 2x in write performance and ~10-20% in read).  

Has anyone had a similar experience?

glusterfs 3.4.0 built on Jul 12 2013 20:36:46
Repository revision: git://git.gluster.com/glusterfs.git
Copyright (c) 2006-2011 Gluster Inc. <http://www.gluster.com>
GlusterFS comes with ABSOLUTELY NO WARRANTY.
You may redistribute copies of GlusterFS under the terms of the GNU General Public License.

Volume Name: perftest
Type: Replicate
Volume ID: 10578349-a466-497a-b7de-1fcd34f967da
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: rdma
Bricks:
Brick1: cs1-i:/gluster/perftest
Brick2: cs2-i:/gluster/perftest

cs1-i:/perftest.rdma  3.2T   13G  3.2T   1% /tmp/perftest

Run began: Wed Jul 17 08:31:39 2013

Record Size 1024 KB
File size set to 5242880 KB
Command line used: iozone -i0 -i1 -r 1024#k -s 5#g
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkwd record stride 
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
5242880 1024 422514 496985 364946 324213



Volume Name: perftest
Type: Replicate
Volume ID: 10578349-a466-497a-b7de-1fcd34f967da
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: cs1-i:/gluster/perftest
Brick2: cs2-i:/gluster/perftest

cs1-i:/perftest.tcp  3.2T   13G  3.2T   1% /tmp/perftest


Run began: Wed Jul 17 08:38:23 2013

Record Size 1024 KB
File size set to 5242880 KB
Command line used: iozone -i0 -i1 -r 1024#k -s 5#g
Output is in Kbytes/sec
Time Resolution = 0.000001 seconds.
Processor cache size set to 1024 Kbytes.
Processor cache line size set to 32 bytes.
File stride size set to 17 * record size.
random random bkwd record stride 
KB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread
5242880 1024 557479 549629 667576 666251


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130717/8e0d2b35/attachment.html>