[Gluster-devel] Automated split-brain resolution

[Joe Julian]

Thinking about it more, I'd still rather have this functionality exposed 
at the client through xattrs. For 5 years I've thought about this, and 
the more I encounter split-brain, the more I think this is the needed 
approach.

"getfattr -n trusted.glusterfs.stat" returns 
xml/json/some_madeup_datastructure with the results of stat from each brick
"getfattr -n trusted.glusterfs.afr" returns the afr matrix
"setfattr -n trusted.glusterfs.sb-pick -v "server2:/srv/brick1"

That gives us the tools we need to choose what to do with any given 
split-brain. For large swaths of automated repair, we can use find.

I suppose that last bit could still be implemented through that cli command.


On 08/07/2014 01:35 AM, Ravishankar N wrote:
>
> Manual resolution of split-brains [1] has been a tedious task 
> involving understanding and modifying AFR's changelog extended 
> attributes. To simplify and to an extent automate this task, we are 
> proposing a new CLI command with which the user can  specify what the 
> source brick/file is, and automatically heal the files in the 
> appropriate direction.
>
> Command: gluster volume resolve-split-brain <VOLNAME> {<bigger_file>  
> |  source-brick <brick_name> [<file>] }
>
> Breaking up the command into its possible options, we have:
>
> a) gluster volume resolve-split-brain <VOLNAME> <bigger_file>
> When this command is executed, AFR will consider the brick having the 
> highest file size as the source and heal it to all other bricks 
> (including all other sources and sinks) in that replica subvolume. If 
> the file size is same in all the bricks, it does *not* heal the file.
>
> b) gluster volume resolve-split-brain <VOLNAME > source-brick 
> <brick_name > [<file>]
>
> When this command is executed, if <file> is specified, AFR heals the 
> file from the source-brick <brick_name> to all other bricks of that 
> replica subvolume. For resolving multiple files, the command must be 
> run iteratively, once per file.
> If <file> is not specified, AFR heals all the files that have an entry 
> in .glusterfs/indices/xattrop *and* are in split-brain. As before, 
> heals happen from source-brick <brick_name> to all other bricks.
>
> Future work could also include extending the command to add other 
> policies like choosing the file having the latest mtime as the source, 
> integration with trash xlator wherein the files deleted from the sink 
> are moved to the trash dir etc.
>
> Please give feedback on the above.
>
> Regards,
> Ravi
>
> [1] https://github.com/gluster/glusterfs/blob/master/doc/split-brain.md
>
>
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://supercolony.gluster.org/pipermail/gluster-devel/attachments/20140808/7b494c3e/attachment.html>