[Gluster-users] performance problem with large directory
raghavendra.hg at gmail.com
Mon Sep 8 04:29:51 UTC 2008
Comments are inlined.
On Mon, Sep 8, 2008 at 3:11 AM, Keith Freedman <freedman at freeformit.com>wrote:
> I have an application which stores old copies of files whenever new
> ones are uploaded.
> it does this by checking if file_exists in the archive folder
> this has roughly 30000 items in it.
How are you checking whether a file exists or not? do you use ls? Can you
make sure, that ls doesn't send a stat call on each of the files in the
directory (as in ls -l )? if ls just sends readdir calls and still the
performance is low, let us know about it.
If you need stat calls, you might need stat-prefetch, which is currently not
> I've got an AFR configuration.
> so, when someone uploads 10 files, it checks for the file existing,
> which does a directory scan which causes afr to check the other
> server, however this takes a reallllly long time.
> and it does this for each file, which is a problem.
> which performance translator would help the most with this.
> the AFR configuration uses local read-volume.
> will any performance translators help given it's an AFR config? I
> understand afr wants to maintain the directory integrity, but it's
> kind of irritating.
> the directory filesize is 2.2 mb so I'm not sure if it's actually
> moving this volume of data constantly or not, but it's painfully slow
> and things seem to timeout constantly.
> any thoughts are appreciated.
> Gluster-users mailing list
> Gluster-users at gluster.org
A centipede was happy quite, until a toad in fun,
Said, "Prey, which leg comes after which?",
This raised his doubts to such a pitch,
He fell flat into the ditch,
Not knowing how to run.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gluster-users