[Gluster-users] How many subfolders in parent folders ?
hans at shapeways.com
Wed May 8 08:10:21 UTC 2013
alain meunier wrote on 20130507:
> I will have a lot (many hundreds of millions) of subdirectories inside a directory, named with md5 hashes.
> I thought of a subdividivision abcd/efgh/... or maybe a/b/c/...
> But it will create a bunch of subfolders.
> Will gluster handler those files without problems ?
Yes, but slow. Very slow IMO :-/ To give you an idea : a recursive walk
through my Distributed-Replicate 14 x 2 filesystem (2M directories, max
100 per subdirectory) takes several days.
> One last thing : hardlinks will be involved to point the files and often deleted.
> Does someone has any experience of the troubleshooting I will have to face ?
Should be fine. Try to avoid filename renames as that causes a DHT miss
My advice : first build a test setup.
Hans Lambermont | Senior Architect
(t) +31407370104 (w) www.shapeways.com
More information about the Gluster-users