[Gluster-users] performance problem with large directory

[Keith Freedman]

I just wanted to report back.

I'm now running 1.4pre5 on this set of AFRed servers
it seems since the installation it's doing a LOT less chatter.
I haven't done any thorough testing but the quick tests I did seems 
to indicate this situation is much improved.

I'll report back if I find this not to be the case with 1.4

Keith

At 09:29 PM 9/7/2008, Raghavendra G wrote:
>Hi Keith,
>
>Comments are inlined.
>
>On Mon, Sep 8, 2008 at 3:11 AM, Keith Freedman 
><<mailto:freedman at freeformit.com>freedman at freeformit.com> wrote:
>I have an application which stores old copies of files whenever new
>ones are uploaded.
>
>it does this by checking if file_exists in the archive folder
>this has roughly 30000 items in it.
>
>
>How are you checking whether a file exists or not? do you  use  ls? 
>Can you make sure, that  ls doesn't  send a stat call on each of the 
>files in the directory (as in ls -l )? if ls just sends readdir 
>calls and still the performance is low, let us know about it.
>
>If you need stat calls, you might need stat-prefetch, which is 
>currently not funcitonal.
>
>
>
>I've got an AFR configuration.
>
>so, when someone uploads 10 files, it checks for the file existing,
>which does a directory scan which causes afr to check the other
>server, however this takes a reallllly long time.
>and it does this for each file, which is a problem.
>
>which performance translator would help the most with this.
>
>the AFR configuration uses local read-volume.
>
>will any performance translators help given it's an AFR config?  I
>understand afr wants to maintain the directory integrity, but it's
>kind of irritating.
>
>the directory filesize is 2.2 mb so I'm not sure if it's actually
>moving this volume of data constantly or not, but it's painfully slow
>and things seem to timeout constantly.
>
>any thoughts are appreciated.
>
>Keith
>
>
>_______________________________________________
>Gluster-users mailing list
><mailto:Gluster-users at gluster.org>Gluster-users at gluster.org
>http://zresearch.com/cgi-bin/mailman/listinfo/gluster-users
>
>
>
>
>--
>Raghavendra G
>
>A centipede was happy quite, until a toad in fun,
>Said, "Prey, which leg comes after which?",
>This raised his doubts to such a pitch,
>He fell flat into the ditch,
>Not knowing how to run.
>-Anonymous