[Gluster-users] Rebalancing with no new bricks.

[Jeff Darcy]

On 06/11/2013 05:55 PM, Harry Mangalam wrote:
> I understand that gluster does not allow gluster-rebalancing in the same
> config (ie, without adding a brick), but would moving some offending
> files to another fs and then copying them back tend to rebalance the FS
> by distributing the copied files more evenly?  

The problem is that no files will move unless the layout (the hash-range
information used to place files) changes, and the layout won't change
unless the set of bricks does, so a rebalance without adding/removing
bricks would end up being a no-op.  However, there are two other ways
this kind of imbalance can be addressed.

Firstly, we can observe that GlusterFS will still find files even if
they're in the "wrong" place according to the layout.  It is possible to
move files manually from one brick to another, so long as appropriate
care is taken e.g. to ensure that all extended attributes are preserved.
 The problem with this approach is that such careful placement might be
undone the next time you do a rebalance for other reasons.

The second approach avoids that problem, but is a bit trickier in other
ways.  There is a feature that allows a user to define their own layout
for a directory, so that it will be left alone by rebalance.  There's
even a script in your friendly neighborhood source tree
(extras/rebalance.py) that will use this to set a layout that's weighted
according to the size of bricks (or free space) instead of just treating
all as equal.  You can use that as an example of how to construct and
apply a valid user-defined layout, plus other techniques that are likely
to be useful.  Lastly, we have a feature that setting the fake
"distribute.migrate-data" xattr on a file will cause use to re-evaluate
where it should go and migrate it to the correct brick if necessary.
Since you seem to be dealing with a relatively small number of files
that need to be moved, it shouldn't be too hard to combine this little
bag of tricks into a solution that meets your needs.  Just let me know
if you'd like me to assist.