[Gluster-users] speeding up getdents()/stat() and/or general perf tuning...

[Mark Mielke]

On 09/29/2009 07:14 PM, mki-glusterfs at mozone.net wrote:
> Hi
>
> I've been noticing some long delays on doing a simple `ls' in
> directories that haven't been recently accessed on a test glusterfs
> system we've put together.  The system(s) consists of a 4 node
> DHT + AFR (x1) setup, running 2.0.6 all with 10GbE connectivity
> between the nodes (and no there is no network bottleneck here as
> iperf proves that the throughput between the machines is ~9Gbps).
>    


I would suggest following advice in another thread:

>   http://www.gluster.com/community/documentation/index.php/Translators/cluster/distribute
> >
> >  It seems to suggest that 'lookup-unhashed' says that the default is 'on'.
> >
> >  Perhaps try turning it 'off'?
>    
> Wei,
>     There are two things we would like you to try. First is what Mark
> has just pointed, the 'option lookup-unhashed off' in distribute. The
> second is 'option transport.socket.nodelay on' in each of your
> protocol/client_and_  protocol/server volumes. Do let us know what
> influence these changes have on your performance.
>
> Avati
>    

The TCP_NODELAY seems particularly relevant to me if many small requests 
are being issued in sequence as a /bin/ls is likely to do?

The lookup-unhashed might be relevant to stat() calls issued as a part 
of the /bin/ls process.

I've been hitting 'ls' problems myself on another system with NFS and 
AutoFS where we have a directory with many symlinks in it. The 'ls' does 
a stat() on each of the symlinks, and in the case of auto-mount - it can 
take a while... :-)

There is a stat-prefetch module that I do not see documentation for. I 
wish there was more comments. A quick skim of it suggests that it 
*might* be designed to improve /bin/ls performance. That it's not 
documented may mean it is for 2.1 or later?

Cheers,
mark


-- 

Mark Mielke<mark at mielke.cc>