[Gluster-users] Glusterfs NFS performance issue

[Pavel Riha]

Dne 03/03/16 08:10, Jiffin Tony Thottan napsal(a):
> On 29/02/16 15:25, Pavel Riha wrote:
>> Hi all,
>>
>> I have read some recent post about performance issues, complaining
>> about the fuse driver and recomended NFS..
>>
>> although my final goal is replicate volume, I'm now just doing some
>> test for reference.
>>
>> my basic benchmark is
>> dd if=/dev/zero of=ddtest bs=1M count=1000 conv=fsync
>>
>> running directly on server xfs partition give me 120-130 MB/s .. OK
>>
>> on client NFS mount using _KERNEL_ nfs server give me 100-112MB/s .. OK
>>
>> but a pure "nfs replacement" gluster config (one brick distribute)
>> give me sometimes 112 and sometimes only 64 !!!  any idea why so
>> unstable?
>> I was watching top,iotop and iftop .. and no idea, the resources were
>> free
>>
>> but the real problem comes with my second benchmark, untar the kernel
>> sources
>> tar xjf /usr/portage/distfiles/linux-3.2.tar.bz2
>>
>> on server localy  .. 16sec
>> kernel nfs mount  .. 1:30
>> gluster nfs mount .. 23min !!! what???
>>
>> is this the small files issue? so much? even on nfs?
>>
>>
>> my last quick test with gluster native (fuse) mount, give me
>> 46MB/s for dd (really slow for non replicate) and
>> 3:58 for the sources untar (not the best, but much much better than
>> gluster nfs)
>>
>>
>> I have done no tunning at all yet. But I thought I will get much
>> better numbers for the one brick distribute. And expected tuning with
>> the replica..
>>
>>
>> server is CPU E5-1650 v3 @ 3.50GHz with 4GB RAM, 7200rpm hdd
>> client is old i5 661 @ 3.33GHz, 4GB RAM
>> gigabit ethernet
>> glusterfs-3.7.4
>> XFS filesystem on brick partition
>>
>
> Can u provide more details about volume configuration (gluster vol info) ?

volume created just with
gluster volume create gtest 1u:/mnt/data/brick/
and run the benchmarks without any tunnig..
/*
if I remember correctly, even I haven't set anything, the gluster info shows
Options Reconfigured:
performance.readdir-ahead: on
*/


then I tried some options, someone metionet here in last posts, a then back.
(without noticable change yet)

current volume info is:

Volume Name: gtest
Type: Distribute
Volume ID: 95cd728e-0b62-4e75-bf18-a0168ec2d0c8
Status: Started
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: 1u:/mnt/data/brick
Options Reconfigured:
performance.nfs.write-behind: On
performance.write-behind: On
performance.readdir-ahead: on
performance.nfs.io-cache: off



Pavel



-- 
Býváme v lásce nespravedliví, protože toho druhého pokládáme za dokonalého.
		-- Jean Paul