[Gluster-devel] Doing LS with a lot of directory, files
Einar Gautun
einar.gautun at statkart.no
Thu Apr 24 11:33:26 UTC 2008
Hello,
I've set up a split network, with one switch(Gbit) having the official
ip for ssh, ntp, nis and so on - mtu 1500. Then I have another
switch(Gbit) with 10.0.0.0/24 - mtu 9000 - for the file transport. Here
I use trunking. This gave so much better responce with ls, so now I have
3029 files in a directory and ls -l takes only 2-3 seconds(test setup).
Even under load this setup works just so much better, going from ls -l
with up to 1 min responce to 1-2 seconds under same load and same
equipment(running system). I have ext3 as filesystem, and io-threads
only on server, unify on client, no other translators serverside or
clientside.
Regards,
Einar
On Thu, 2008-04-24 at 12:47 +0200, Tom Myny wrote:
> Hello,
>
> I'm running afr on two storage servers, with three clients.
> For the moment, we have copied over 500 million small files on it, splitting
> into each directory which contains 1000 files.
>
> When doing ls in directory containing 1000 directory's we have the following
> issue:
>
>
> - Ls is taking more then 15 minutes to complete in a directory with 1000
> folders. (this will be split also to 100 folders later, but it's now a big
> problem)
> -> Yes, for now its ls --color=auto by default on debian :D
> - When doing copies from other clients, those copies halt until that ls is
> complete.
>
>
> Is there a way to
>
> 1) Do a ls faster (ok, I know it can be that fast like on the filesystem
> itself, but on the filesystem (or an nfs system) it takes max 15 seconds)
> 2) When someone is doing an ls, the other processes are not freesing.
> (checking on the storage servers, we have a load of 0.00)
>
> The filesystems we use are based on xfs.
> An example of a server config:
>
> volume sas-ds
> type storage/posix
> option directory /sas/data
> end-volume
>
> volume sas-ns
> type storage/posix
> option directory /sas/ns
> end-volume
>
> volume sata-ds
> type storage/posix
> option directory /sata/data
> end-volume
>
> volume sata-ns
> type storage/posix
> option directory /sata/ns
> end-volume
>
> volume sas-backup-ds
> type protocol/client
> option transport-type tcp/client
> option remote-host x.x.x.x
> option remote-subvolume sas-ds
> end-volume
>
> volume sas-backup-ns
> type protocol/client
> option transport-type tcp/client
> option remote-host x.x.x.x
> option remote-subvolume sas-ns
> end-volume
>
> ...
>
> volume sas-unify
> type cluster/unify
> subvolumes sas-ds-afr
> option namespace sas-ns-afr
> option scheduler rr
> end-volume
>
> volume sata-unify
> type cluster/unify
> subvolumes sata-ds-afr
> option namespace sata-ns-afr
> option scheduler rr
> end-volume
>
> volume sas
> type performance/io-threads
> option thread-count 16
> option cache-size 256MB
> subvolumes sas-unify
> end-volume
>
> volume sata
> type performance/io-threads
> option thread-count 16
> option cache-size 256MB
> subvolumes sata-unify
> end-volume
>
> ..
>
> I hope to fix this, because we want to double this next year :)
>
>
> Regards,
> Tom
>
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at nongnu.org
> http://lists.nongnu.org/mailman/listinfo/gluster-devel
--
Einar Gautun einar.gautun at statkart.no
Statens kartverk | Norwegian Mapping Authority
3507 Hønefoss | NO-3507 Hønefoss, Norway
Ph +47 32118372 Fax +47 32118101 Mob +47 92692662
More information about the Gluster-devel
mailing list