[Gluster-users] How many subfolders in parent folders ?

[alain meunier]

Hi Hans !

Do you think that an hexadecimal way could do fine ?

a/b/c/d/e/f/0/1/2...

That way : 16 subfolders per directory.

Will that be fine (performance wise) ?

Many thanks !

Regards,

Alain 
> Date: Wed, 8 May 2013 10:10:21 +0200
> From: hans at shapeways.com
> To: deco33 at hotmail.fr
> CC: gluster-users at gluster.org
> Subject: Re: [Gluster-users] How many subfolders in parent folders ?
> 
> Hi Alain,
> 
> alain meunier wrote on 20130507:
> > I will have a lot (many hundreds of millions) of subdirectories inside a directory, named with md5 hashes.
> > I thought of a subdividivision abcd/efgh/... or maybe a/b/c/...
> > But it will create a bunch of subfolders.
> > Will gluster handler those files without problems ?
> 
> Yes, but slow. Very slow IMO :-/ To give you an idea : a recursive walk
> through my Distributed-Replicate 14 x 2 filesystem (2M directories, max
> 100 per subdirectory) takes several days.
> 
> > One last thing : hardlinks will be involved to point the files and often deleted.
> > Does someone has any experience of the troubleshooting I will have to face ?
> 
> Should be fine. Try to avoid filename renames as that causes a DHT miss
> penalty.
> 
> My advice : first build a test setup.
> 
> -- Hans
> -- 
> Hans Lambermont | Senior Architect
> (t) +31407370104 (w) www.shapeways.com
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-users/attachments/20130508/d8d2de45/attachment.html>