[Gluster-users] fuse client performance issues

[Daniel Davidson]

Hello,

I am preparing our first real production gluster service and I have run 
in to a performance issue with the fuse client.

Here is our config:

Volume Name: biocluster-home
Type: Distribute
Volume ID: a9352088-d9d9-4438-8a87-5bbaa0cded5e
Status: Started
Number of Bricks: 3
Transport-type: tcp
Bricks:
Brick1: gl-0:/poola/brick
Brick2: gl-0:/poolb/brick
Brick3: gl-0:/poolc/brick
Options Reconfigured:
diagnostics.latency-measurement: on
diagnostics.count-fop-hits: on

The bricks are all ZFS volumes.

When we transfer to the ZFS volumes on the storage node we get a great 
speed:

./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F 
/poola/f{0,1,2,3,4,5,6,7,8}.ioz |grep Children
         Children see throughput for  8 initial writers  = 2200360.69 kB/sec

When we go to the fuse client on the gluster storage node, we get good 
speed:

./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F 
/glusterhome/f{0,1,2,3,4,5,6,7,8}.ioz |grep Children
         Children see throughput for  8 initial writers  =  803187.06 kB/sec

When we go to the fuse client on another system, it tanks:

./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F 
/gluster/f{0,1,2,3,4,5,6,7,8}.ioz |grep Children
         Children see throughput for  8 initial writers  =  114423.71 kB/sec

However the NFS client still does ok:

./iozone -w -c -e -i 0 -+n -C -r 64k -s 1g -t 8 -F 
/glusternfs/f{0,1,2,3,4,5,6,7,8}.ioz |grep Children
         Children see throughput for  8 initial writers  =  952116.27 kB/sec

Iperf shows the network is ok:

[  3]  0.0-10.0 sec  11.5 GBytes  9.83 Gbits/sec

Am I missing anything here?  Is there anything I can do to improve the 
performance of the cluster fuse client?

Dan