[Gluster-devel] Feature review: Improved rebalance performance

[Raghavendra Gowdappa]

----- Original Message -----
> From: "Xavier Hernandez" <xhernandez at datalab.es>
> To: "Shyamsundar Ranganathan" <srangana at redhat.com>
> Cc: gluster-devel at gluster.org
> Sent: Monday, June 30, 2014 2:30:01 PM
> Subject: Re: [Gluster-devel] Feature review: Improved rebalance performance
> 
> Hi Shyam,
> 
> On Thursday 26 June 2014 14:41:13 Shyamsundar Ranganathan wrote:
> > It also touches upon a rebalance on access like mechanism where we could
> > potentially, move data out of existing bricks to a newer brick faster, in
> > the case of brick addition, and vice versa for brick removal, and heal the
> > rest of the data on access.
> > 
> Will this "rebalance on access" feature be enabled always or only during a
> brick addition/removal to move files that do not go to the affected brick
> while the main rebalance is populating or removing files from the brick ?
> 
> I like all the proposed ideas. I think they would improve the performance of
> the rebalance operation considerably. Probably we will need to define some
> policies to limit the amount of bandwidth that rebalance is allowed to use
> and
> at which hours, but this can be determined later.
> 
> I would also consider using index or changelog xlators to track renames and
> let rebalance consume it. Currently a file or directory rename makes that
> files correctly placed in the right brick need to be moved to another brick.
> A
> full rebalance crawling all the file system seems too expensive for this kind
> of local changes (the effects of this are orders of magnitude smaller than
> adding or removing a brick). Having a way to list pending moves due to rename
> without scanning all the file system would be great.
> 
> Another thing to consider for future versions is to modify the current DHT to
> a consistent hashing and even the hash value (using gfid instead of a hash of
> the name would solve the rename problem). The consistent hashing would
> drastically reduce the number of files that need to be moved and already
> solves some of the current problems. This change needs a lot of thinking
> though.

The problem with using gfid for hashing instead of name is that we run into a chicken and egg problem. Before lookup, we cannot know the gfid of the file and to lookup the file, we need gfid to find out the node in which file resides. Of course, this problem would go away if we lookup (may be just during fresh lookups) on all the nodes, but that slows down the fresh lookups and may not be acceptable.

> 
> Xavi
> 
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
>