[Gluster-users] Tuning for small files

[Ben Turner]

----- Original Message -----
> From: hmlth at t-hamel.fr
> To: abauer at magix.net
> Cc: gluster-users at gluster.org
> Sent: Monday, September 28, 2015 7:40:52 AM
> Subject: Re: [Gluster-users] Tuning for small files
> 
> I'm also quite interested by small files performances optimization, but
> I'm a bit confused about the best option between 3.6/3.7.
> 
> Ben Turner was saying that 3.6 might give the best performances:
> http://www.gluster.org/pipermail/gluster-users/2015-September/023733.html
> 
> What kind of gain is expected (with consistent-metadata) if this
> regression is solved?

Just to be clear, the issue I am talking about is metadata only(think ls -l or file browsing).  It doesn't affect small file perf(well not that much, I'm sure a little, but I have never quantified it), with server and client event threads set to 4 + lookup optimize I see between a 200-300% gain on my systems on 3.7 vs 3.6 builds.  If I needed fast metadata I would go with 3.6, if I need fast smallfile I would go with 3.7.  If I needed both I would pick the less of the two evils and go with that one and upgrade when the fix is released.

-b


> 
> I tried 3.6.5 (last version for debian jessie), and it's a bit better
> than 3.7.4 but not by much (10-15%).
> 
> I was also wondering if there is recommendations for the underlying file
> system of the bricks (xfs, ext4, tuning...).
> 
> 
> Regards
> 
> Thomas HAMEL
> 
> On 2015-09-28 12:04, André Bauer wrote:
> > If you're not already on Glusterfs 3.7.x i would recommend an update
> > first.
> > 
> > Am 25.09.2015 um 17:49 schrieb Thibault Godouet:
> >> Hi,
> >> 
> >> There are quite a few tuning parameters for Gluster (as seen in
> >> Gluster
> >> volume XYZ get all), but I didn't find much documentation on those.
> >> Some people do seem to set at least some of them, so the knowledge
> >> must
> >> be somewhere...
> >> 
> >> Is there a good source of information to understand what they mean,
> >> and
> >> recommendation on how to set them to get a good small file
> >> performance?
> >> 
> >> Basically what I'm trying to optimize is for svn operations (e.g. svn
> >> checkout, or svn branch) on a replicated 2 x 1 volume (hosted on 2
> >> VMs,
> >> 16GB ram, 4 cores each, 10Gb/s network tested at full speed), using a
> >> NFS mount which appears much faster than fuse in this case (but still
> >> much slower than when served by a normal NFS server).
> >> Any recommendation for such a setup?
> >> 
> >> Thanks,
> >> Thibault.
> >> 
> >> 
> >> 
> >> _______________________________________________
> >> Gluster-users mailing list
> >> Gluster-users at gluster.org
> >> http://www.gluster.org/mailman/listinfo/gluster-users
> >> 
> > 
> > 
> > --
> > Mit freundlichen Grüßen
> > André Bauer
> > 
> > MAGIX Software GmbH
> > André Bauer
> > Administrator
> > August-Bebel-Straße 48
> > 01219 Dresden
> > GERMANY
> > 
> > tel.: 0351 41884875
> > e-mail: abauer at magix.net
> > abauer at magix.net <mailto:Email>
> > www.magix.com <http://www.magix.com/>
> > 
> > 
> > Geschäftsführer | Managing Directors: Dr. Arnd Schröder, Michael Keith
> > Amtsgericht | Commercial Register: Berlin Charlottenburg, HRB 127205
> > 
> > Find us on:
> > 
> > <http://www.facebook.com/MAGIX> <http://www.twitter.com/magix_de>
> > <http://www.youtube.com/wwwmagixcom> <http://www.magixmagazin.de>
> > ----------------------------------------------------------------------
> > The information in this email is intended only for the addressee named
> > above. Access to this email by anyone else is unauthorized. If you are
> > not the intended recipient of this message any disclosure, copying,
> > distribution or any action taken in reliance on it is prohibited and
> > may be unlawful. MAGIX does not warrant that any attachments are free
> > from viruses or other defects and accepts no liability for any losses
> > resulting from infected email transmissions. Please note that any
> > views expressed in this email may be those of the originator and do> > Gluster-users mailing list
> > Gluster-users at gluster.org
> > http://www.gluster.org/mailman/listinfo/gluster-users
> 
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users