[Gluster-users] small files and cluster/stripe

Jeff Anderson-Lee jonah at eecs.berkeley.edu
Thu May 13 19:36:58 UTC 2010


cluster/stripe will split large files across multiple volumes, but it 
seems to
always put the first part of the file on the first volume; if you have a 
bunch of small files they all end up there, and one volume gets heavily 
used by small files while the others are empty.

cluster/distribute spreads files across multiple volumes, but it puts 
the whole file on a single volume.

Some marriage of the two would be helpful for workloads which contain 
both large and small files, like adding an "option block-size ..." to 
cluster/distribute or "option distribute" to cluster/stripe; it would 
use the filename hash modulo nSubvolumes to determine which volume to 
start in for the first block, then rotate around the stripe for the rest.

I suppose I can work-around by creating multiple volumes as 
sub-directories of the same partition, then striping across those in 
rotations, and distributing across the stripes.

Is there some other way?  Am I missing something?

Jeff Anderson-Lee




More information about the Gluster-users mailing list