[Gluster-users] Unable to self-heal permissions/ownership of '/' (possible split-brain)

[Dan Bretherton]

On 25/11/10 10:55, Dan Bretherton wrote:
> Hello all-
>
> I have set up two new servers each with two RAID 5 arrays.  I am trying
> to distribute the load across the two RAID arrays on each server by
> creating a distributed, replicated volume.  This is the output from
> "gluster volume info".
>
> Volume Name: marine2
> Type: Distributed-Replicate
> Status: Started
> Number of Bricks: 2 x 2 = 4
> Transport-type: tcp
> Bricks:
> Brick1: 192.171.166.86:/vg1lv0/glusterfs
> Brick2: 192.171.166.87:/vg1lv0/glusterfs
> Brick3: 192.171.166.86:/vg2lv0/glusterfs
> Brick4: 192.171.166.87:/vg2lv0/glusterfs
> Options Reconfigured:
> performance.quick-read: off
>
> The volume seems to be working normally, but I am seeing the following
> messages repeated in my volume log file:
>
> [2010-11-25 10:27:06.867368] I [afr-common.c:672:afr_lookup_done]
> marine2-replicate-0: split brain detected during lookup of /.
> [2010-11-25 10:27:06.867454] I [afr-common.c:716:afr_lookup_done]
> marine2-replicate-0: background  meta-data data self-heal triggered. path: /
> [2010-11-25 10:27:06.867614] I [afr-common.c:672:afr_lookup_done]
> marine2-replicate-1: split brain detected during lookup of /.
> [2010-11-25 10:27:06.867633] I [afr-common.c:716:afr_lookup_done]
> marine2-replicate-1: background  meta-data data self-heal triggered. path: /
> [2010-11-25 10:27:06.868174] E
> [afr-self-heal-metadata.c:524:afr_sh_metadata_fix] marine2-replicate-0:
> Unable to self-heal permissions/ownership of '/' (possible split-brain).
> Please fix the file on all backend volumes
> [2010-11-25 10:27:06.868265] E
> [afr-self-heal-metadata.c:524:afr_sh_metadata_fix] marine2-replicate-1:
> Unable to self-heal permissions/ownership of '/' (possible split-brain).
> Please fix the file on all backend volumes
> [2010-11-25 10:27:06.868483] I
> [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk]
> marine2-replicate-0: background  meta-data data self-heal completed on /
> [2010-11-25 10:27:06.868541] I
> [afr-self-heal-common.c:1526:afr_self_heal_completion_cbk]
> marine2-replicate-1: background  meta-data data self-heal completed on /
>
> I don't understand why a self heal is ever necessary on "/".  These
> presumably are the brick directories (/vg1lv0/glusterfs etc), which are
> all owned by root and have the same permissions.  Is this something I
> should be worried about, and if not how can I stop these warnings?
> -Dan.
>   

Sorry I have just seen this previous posting while trawling though the
mail archives researching another problem (but I did Google search
before posting mine earlier on):

http://gluster.org/pipermail/gluster-users/2010-November/005780.html

My getfattr output.  There doesn't seem to have been any more discussion
in the previous thread about this issue.

[root at bdan4 ~]# getfattr -d -e hex -m trusted.afr /vg1lv0/glusterfs
getfattr: Removing leading '/' from absolute path names
# file: vg1lv0/glusterfs
trusted.afr.marine2-client-0=0x000000000000000000000000
trusted.afr.marine2-client-1=0x000000000000000100000000

[root at bdan4 ~]# getfattr -d -e hex -m trusted.afr /vg2lv0/glusterfs
getfattr: Removing leading '/' from absolute path names
# file: vg2lv0/glusterfs
trusted.afr.marine2-client-2=0x000000000000000000000000
trusted.afr.marine2-client-3=0x000000000000000100000000

[root at bdan5 ~]#  getfattr -d -e hex -m trusted.afr /vg1lv0/glusterfs
getfattr: Removing leading '/' from absolute path names
# file: vg1lv0/glusterfs
trusted.afr.marine2-client-0=0x000000000000000100000000
trusted.afr.marine2-client-1=0x000000000000000000000000

[root at bdan5 ~]#  getfattr -d -e hex -m trusted.afr /vg2lv0/glusterfs
getfattr: Removing leading '/' from absolute path names
# file: vg2lv0/glusterfs
trusted.afr.marine2-client-2=0x000000000000000100000000
trusted.afr.marine2-client-3=0x000000000000000000000000