[Gluster-users] How many subfolders in parent folders ?

[Hans Lambermont]

Hi Alain,

alain meunier wrote on 20130507:
> I will have a lot (many hundreds of millions) of subdirectories inside a directory, named with md5 hashes.
> I thought of a subdividivision abcd/efgh/... or maybe a/b/c/...
> But it will create a bunch of subfolders.
> Will gluster handler those files without problems ?

Yes, but slow. Very slow IMO :-/ To give you an idea : a recursive walk
through my Distributed-Replicate 14 x 2 filesystem (2M directories, max
100 per subdirectory) takes several days.

> One last thing : hardlinks will be involved to point the files and often deleted.
> Does someone has any experience of the troubleshooting I will have to face ?

Should be fine. Try to avoid filename renames as that causes a DHT miss
penalty.

My advice : first build a test setup.

-- Hans
-- 
Hans Lambermont | Senior Architect
(t) +31407370104 (w) www.shapeways.com