[Gluster-users] Small File and "ls" performance ..

Joshua Baker-LePain jlb17 at duke.edu
Fri Jun 4 15:25:57 UTC 2010

On Wed, 5 May 2010 at 1:00am, Tejas N. Bhise wrote

> We have recently made some code changes in an effort to improve small
> file and 'ls' performance. The patches are -
> selective readdirp - http://patches.gluster.com/patch/3203/
> dht lookup revalidation optimization - http://patches.gluster.com/patch/3204/
> updated write-behind default values - http://patches.gluster.com/patch/3223/
> DISCLAIMER : These patches have not made it to any supported release
> yet and have not been tested yet. Don't use them in production. I am
> providing this information only as some advance notice for those in
> the community who might be interested in trying out these changes and
> provide feedback.

I would like to cast my vote firmly in favor of these patches.  I did a 
bit of a torture test with my scratch gluster setup:

Storage bricks:
10 HP DL160 G5s, each with a single 7200RPM SATA disk

Same hardware.  Everything is connected via GbE to the same switch.

Gluster setup:
Gluster 3.0.4.  Standard replicate then distribute setup created via 

An old version of <http://people.redhat.com/dledford/memtest.shtml>. 
This script unpacks *lots* of copies of the Linux kernel tarball (it's 
based on memory size -- for this client, it was 98), diffs all of them 
against the first copy, and then removes them all.  So, lots of small 

Length of 1 run before patches:

Length of 1 run after patches:

Wow.  And the run with the test patches didn't generate any errors.

> Once these are fully tested they will make to an officially supported release.

I rather look forward to that.

Joshua Baker-LePain
QB3 Shared Cluster Sysadmin

More information about the Gluster-users mailing list