[Gluster-users] Glusterfs NFS performance issue

[Ronny Adsetts]

Pavel Riha wrote on 29/02/2016 09:55:
> 
> I have read some recent post about performance issues, complaining
> about the fuse driver and recomended NFS..
> 
> although my final goal is replicate volume, I'm now just doing some
> test for reference.
> 
> my basic benchmark is dd if=/dev/zero of=ddtest bs=1M count=1000
> conv=fsync
> 
> running directly on server xfs partition give me 120-130 MB/s .. OK
> 
> on client NFS mount using _KERNEL_ nfs server give me 100-112MB/s ..
> OK
> 
> but a pure "nfs replacement" gluster config (one brick distribute)
> give me sometimes 112 and sometimes only 64 !!!  any idea why so
> unstable? I was watching top,iotop and iftop .. and no idea, the
> resources were free
> 
> but the real problem comes with my second benchmark, untar the kernel
> sources tar xjf /usr/portage/distfiles/linux-3.2.tar.bz2
> 
> on server localy  .. 16sec kernel nfs mount  .. 1:30 gluster nfs
> mount .. 23min !!! what???
> 
> is this the small files issue? so much? even on nfs?
> 
> my last quick test with gluster native (fuse) mount, give me 46MB/s
> for dd (really slow for non replicate) and 3:58 for the sources untar
> (not the best, but much much better than gluster nfs)
> 
> I have done no tunning at all yet. But I thought I will get much
> better numbers for the one brick distribute. And expected tuning with
> the replica..
> 
> server is CPU E5-1650 v3 @ 3.50GHz with 4GB RAM, 7200rpm hdd client
> is old i5 661 @ 3.33GHz, 4GB RAM gigabit ethernet glusterfs-3.7.4 XFS
> filesystem on brick partition

Just a quick "me too". Not sure what if 

Using a 2-brick replica over 2x bonded (round robin) 1Gb ethernet. Mount is on one of the brick nodes.

A kernel un-tar seems to really stress Gluster.

Extraction to Gluster NFS mount:

# time tar xf /tmp/linux-4.4.tar

real 13m46.308s
user 0m1.224s
sys 0m19.608s

Extraction to Gluster FUSE mount:

# time tar xf /tmp/linux-4.4.tar

real 9m10.413s
user 0m1.964s
sys 0m16.740s

Extraction to local hardware-RAID backed FS (where the bricks are):

# time tar xf /tmp/linux-4.4.tar

real 0m2.771s
user 0m0.284s
sys 0m2.396s

Ronny

-- 
Ronny Adsetts
Technical Director
Amazing Internet Ltd, London
t: +44 20 8977 8943
w: www.amazinginternet.com

Registered office: 85 Waldegrave Park, Twickenham, TW1 4TJ
Registered in England. Company No. 4042957


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: OpenPGP digital signature
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20160303/3ac16e60/attachment.sig>