[Gluster-users] 3.2.2 Performance Issue

[Joey McDonald]

Hello all,

I've configured 4 bricks over a GigE network, however I'm getting very slow
performance for writing to my gluster share.

Just set this up this week, and here's what I'm seeing:

[root at vm-container-0-0 ~]# gluster --version | head -1
glusterfs 3.2.2 built on Jul 14 2011 13:34:25

[root at vm-container-0-0 pifs]# gluster volume info

Volume Name: pifs
Type: Distributed-Replicate
Status: Started
Number of Bricks: 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: vm-container-0-0:/gluster
Brick2: vm-container-0-1:/gluster
Brick3: vm-container-0-2:/gluster
Brick4: vm-container-0-3:/gluster

The 4 systems, are each storage bricks and storage clients, mounting gluster
like so:

[root at vm-container-0-1 ~]# df -h  /pifs/
Filesystem            Size  Used Avail Use% Mounted on
glusterfs#127.0.0.1:pifs
                      1.8T  848M  1.7T   1% /pifs

iperf show's network through put looking good:

[root at vm-container-0-0 pifs]# iperf -c vm-container-0-1
------------------------------------------------------------
Client connecting to vm-container-0-1, TCP port 5001
TCP window size: 16.0 KByte (default)
------------------------------------------------------------
[  3] local 10.19.127.254 port 53441 connected with 10.19.127.253 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes    941 Mbits/sec


Then, writing to the local disk is pretty fast:

[root at vm-container-0-0 pifs]# dd if=/dev/zero of=/root/dd_test.img bs=1M
count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 4.8066 seconds, 436 MB/s

However, writes to the gluster share, are abysmally slow:

[root at vm-container-0-0 pifs]# dd if=/dev/zero of=/pifs/dd_test.img bs=1M
count=2000
2000+0 records in
2000+0 records out
2097152000 bytes (2.1 GB) copied, 241.866 seconds, 8.7 MB/s

Other than the fact that it's quite slow, it seems to be very stable.

iozone testing shows about the same results.

Any help troubleshooting would be much appreciated. Thanks!

   --joey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20110809/a2c80139/attachment.html>