[Gluster-devel] Automated split-brain resolution

[Joe Julian]

On 08/07/2014 03:08 AM, Niels de Vos wrote:
> On Thu, Aug 07, 2014 at 03:17:11PM +0530, Ravishankar N wrote:
>> On 08/07/2014 03:06 PM, Niels de Vos wrote:
>>> On Thu, Aug 07, 2014 at 02:05:34PM +0530, Ravishankar N wrote:
>>>> Manual resolution of split-brains [1] has been a tedious task
>>>> involving understanding and modifying AFR's changelog extended
>>>> attributes. To simplify and to an extent automate this task, we are
>>>> proposing a new CLI command with which the user can  specify  what
>>>> the source brick/file is, and automatically heal the files in the
>>>> appropriate direction.
>>> What about automatically healing directories that are in split-brain?
>> Conservative merge happens when possible while healing directories,
>> but yes gfid split-brain  (one brick having a file with gfid-g1 and
>> the other having a directory with the same gfid-g1) won't be
>> resolved with this command. The reason why are not wanting to it
>> right now is resolving such split-brains involves
>> unlinking/rmdir'ing one of the entries and is best when we integrate
>> with trash xlator.
> Okay, good to know. Thanks!
> Niels
Wait, directories *are* supposed to automatically heal from split-brain? 
Guess I need to file a bug report. That doesn't happen. All the metadata 
and gfid can be the same, but since the trusted.afr are both dirty, 
it'll stay split-brain forever.
>
>>> Thanks,
>>> Niels
>>>
>>>> Command: gluster volume resolve-split-brain <VOLNAME> {<bigger_file>
>>>> |  source-brick <brick_name> [<file>] }
>>>>
>>>> Breaking up the command into its possible options, we have:
>>>>
>>>> a) gluster volume resolve-split-brain <VOLNAME> <bigger_file>
>>>> When this command is executed, AFR will consider the brick having
>>>> the highest file size as the source and heal it to all other bricks
>>>> (including all other sources and sinks) in that replica subvolume.
>>>> If the file size is same in all the bricks, it does *not* heal the
>>>> file.
>>>>
>>>> b) gluster volume resolve-split-brain <VOLNAME > source-brick
>>>> <brick_name > [<file>]
>>>>
>>>> When this command is executed, if <file> is specified, AFR heals the
>>>> file from the source-brick <brick_name> to all other bricks of that
>>>> replica subvolume. For resolving multiple files, the command must be
>>>> run iteratively, once per file.
>>>> If <file> is not specified, AFR heals all the files that have an
>>>> entry in .glusterfs/indices/xattrop *and* are in split-brain. As
>>>> before, heals happen from source-brick <brick_name> to all other
>>>> bricks.
>>>>
>>>> Future work could also include extending the command to add other
>>>> policies like choosing the file having the latest mtime as the
>>>> source, integration with trash xlator wherein the files deleted from
>>>> the sink are moved to the trash dir etc.
>>>>
>>>> Please give feedback on the above.
>>>>
>>>> Regards,
>>>> Ravi
>>>>
>>>> [1] https://github.com/gluster/glusterfs/blob/master/doc/split-brain.md
>>>> _______________________________________________
>>>> Gluster-devel mailing list
>>>> Gluster-devel at gluster.org
>>>> http://supercolony.gluster.org/mailman/listinfo/gluster-devel
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://supercolony.gluster.org/mailman/listinfo/gluster-devel