[Gluster-users] speeding up getdents()/stat() and/or general perf tuning...
mark at mark.mielke.cc
Wed Sep 30 00:53:44 UTC 2009
On 09/29/2009 07:14 PM, mki-glusterfs at mozone.net wrote:
> I've been noticing some long delays on doing a simple `ls' in
> directories that haven't been recently accessed on a test glusterfs
> system we've put together. The system(s) consists of a 4 node
> DHT + AFR (x1) setup, running 2.0.6 all with 10GbE connectivity
> between the nodes (and no there is no network bottleneck here as
> iperf proves that the throughput between the machines is ~9Gbps).
I would suggest following advice in another thread:
> > It seems to suggest that 'lookup-unhashed' says that the default is 'on'.
> > Perhaps try turning it 'off'?
> There are two things we would like you to try. First is what Mark
> has just pointed, the 'option lookup-unhashed off' in distribute. The
> second is 'option transport.socket.nodelay on' in each of your
> protocol/client_and_ protocol/server volumes. Do let us know what
> influence these changes have on your performance.
The TCP_NODELAY seems particularly relevant to me if many small requests
are being issued in sequence as a /bin/ls is likely to do?
The lookup-unhashed might be relevant to stat() calls issued as a part
of the /bin/ls process.
I've been hitting 'ls' problems myself on another system with NFS and
AutoFS where we have a directory with many symlinks in it. The 'ls' does
a stat() on each of the symlinks, and in the case of auto-mount - it can
take a while... :-)
There is a stat-prefetch module that I do not see documentation for. I
wish there was more comments. A quick skim of it suggests that it
*might* be designed to improve /bin/ls performance. That it's not
documented may mean it is for 2.1 or later?
Mark Mielke<mark at mielke.cc>
More information about the Gluster-users