[Gluster-users] small files and cluster/stripe
Jeff Anderson-Lee
jonah at eecs.berkeley.edu
Thu May 13 19:36:58 UTC 2010
cluster/stripe will split large files across multiple volumes, but it
seems to
always put the first part of the file on the first volume; if you have a
bunch of small files they all end up there, and one volume gets heavily
used by small files while the others are empty.
cluster/distribute spreads files across multiple volumes, but it puts
the whole file on a single volume.
Some marriage of the two would be helpful for workloads which contain
both large and small files, like adding an "option block-size ..." to
cluster/distribute or "option distribute" to cluster/stripe; it would
use the filename hash modulo nSubvolumes to determine which volume to
start in for the first block, then rotate around the stripe for the rest.
I suppose I can work-around by creating multiple volumes as
sub-directories of the same partition, then striping across those in
rotations, and distributing across the stripes.
Is there some other way? Am I missing something?
Jeff Anderson-Lee
More information about the Gluster-users
mailing list