[Gluster-users] Poor performance of NFS client for large writes compared to native client

[Ben Turner]

----- Original Message -----
> From: "Vijay Bellur" <vbellur at redhat.com>
> To: "Behrooz Shafiee" <shafiee01 at gmail.com>, "Gluster-users at gluster.org List" <gluster-users at gluster.org>
> Sent: Thursday, April 30, 2015 6:44:11 AM
> Subject: Re: [Gluster-users] Poor performance of NFS client for large writes compared to native client
> 
> On 04/30/2015 06:49 AM, Behrooz Shafiee wrote:
> > Hi,
> >
> >   I was comparing GlusterFS native and NFS clients and I noticed, NFS
> > client is significantly slower for large writes. I wrote about 200, 1GB
> > files using a 1MB block sizes and NFS throughput was almost half of
> > native client. Can anyone explain why is that?
> >
> 
> Depending on where the file gets scheduled, NFS might need an additional
> network hop. That can contribute to additional latency and less
> throughput than the native client.

To tag on here GlusterFS mounts use the hash algorithm to know which server to write directly to.  NFS is not aware of this so all files get routed through the server that is mounted just like Vijay said.  The server relaying the file adds this extra hop and contributes to the latency / slowdown.  I estimate performance like:

10G interface with 12 disk RAID 6:

GFS Read(replica 1 or 2) = 720 MB/s
GFS Write(replica 1) = 820 MB/s
GFS Write(replica 2) = 410 MB/s

NFS Read(replica 1 or 2) = 535 MB/s
NFS Write(replica 1) = 400 MB/s
NFS Write(replica 2) = 250 MB/s

So with replica 2 Gluster FS I would expect ~410 MB / sec writes and on the same volume over NFS I would expect 250 MB / sec.  Its not a full 50% but its close.

HTH!

-b


> -Vijay
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users
>