[Gluster-users] Poor performance of NFS client for large writes compared to native client

[Ben Turner]

----- Original Message -----
> From: "Behrooz Shafiee" <shafiee01 at gmail.com>
> To: "Ben Turner" <bturner at redhat.com>
> Cc: "Gluster-users at gluster.org List" <gluster-users at gluster.org>
> Sent: Thursday, April 30, 2015 9:34:31 AM
> Subject: Re: [Gluster-users] Poor performance of NFS client for large writes compared to native client
> 
> Thanks, it clarifies write slowdown! But my reads with NFS are as fast as
> GlusterFS native client. Does it mean the server which NFS was mounted with
> is actually hosting those files so no extra hop and same performance?

Yepo, that is probably the case.  With reads setting read ahead on the brick device is pretty important.  I recommend trying:

    echo 65536 > /sys/block/$device_name/queue/read_ahead_kb

Only use this if you have a RAID, I normally use RAID 6 with 12 disks.

-b

> 
> Thanks,
> On 30 Apr 2015 8:27 am, "Ben Turner" <bturner at redhat.com> wrote:
> 
> > ----- Original Message -----
> > > From: "Vijay Bellur" <vbellur at redhat.com>
> > > To: "Behrooz Shafiee" <shafiee01 at gmail.com>, "Gluster-users at gluster.org
> > List" <gluster-users at gluster.org>
> > > Sent: Thursday, April 30, 2015 6:44:11 AM
> > > Subject: Re: [Gluster-users] Poor performance of NFS client for large
> > writes compared to native client
> > >
> > > On 04/30/2015 06:49 AM, Behrooz Shafiee wrote:
> > > > Hi,
> > > >
> > > >   I was comparing GlusterFS native and NFS clients and I noticed, NFS
> > > > client is significantly slower for large writes. I wrote about 200, 1GB
> > > > files using a 1MB block sizes and NFS throughput was almost half of
> > > > native client. Can anyone explain why is that?
> > > >
> > >
> > > Depending on where the file gets scheduled, NFS might need an additional
> > > network hop. That can contribute to additional latency and less
> > > throughput than the native client.
> >
> > To tag on here GlusterFS mounts use the hash algorithm to know which
> > server to write directly to.  NFS is not aware of this so all files get
> > routed through the server that is mounted just like Vijay said.  The server
> > relaying the file adds this extra hop and contributes to the latency /
> > slowdown.  I estimate performance like:
> >
> > 10G interface with 12 disk RAID 6:
> >
> > GFS Read(replica 1 or 2) = 720 MB/s
> > GFS Write(replica 1) = 820 MB/s
> > GFS Write(replica 2) = 410 MB/s
> >
> > NFS Read(replica 1 or 2) = 535 MB/s
> > NFS Write(replica 1) = 400 MB/s
> > NFS Write(replica 2) = 250 MB/s
> >
> > So with replica 2 Gluster FS I would expect ~410 MB / sec writes and on
> > the same volume over NFS I would expect 250 MB / sec.  Its not a full 50%
> > but its close.
> >
> > HTH!
> >
> > -b
> >
> >
> > > -Vijay
> > >
> > > _______________________________________________
> > > Gluster-users mailing list
> > > Gluster-users at gluster.org
> > > http://www.gluster.org/mailman/listinfo/gluster-users
> > >
> >
>