[Gluster-users] Split brain

[rwecker at ssd.org]

Hi, 

I am having a problem with a split-brain issue that does not seem to match up with documentation on how to solve it. 

gluster volume heal VMData2 info gives: 


Brick found2.ssd.org:/data/brick6/data 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45/82a7027b-321c-4bd9-8afc-2a12cfa23bfc 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45/82a7027b-321c-4bd9-8afc-2a12cfa23bfc.meta 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/ce2831a4-3e82-4bf8-bb68-82afa0c401fe 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images - Is in split-brain 

/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/ee8e0fd3-cf68-4f2a-9d0d-f7ee665cc8c3 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/a4bea667-c65d-4085-9b90-896bf7fc55ff 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/4981580d-628f-4266-8e7e-f7a0bcae2dbb 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/2ee209b4-3e21-47fc-8342-033a37605d65 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/dom_md/xleases 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/dom_md 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/28d32853-5c89-4c5f-8fb7-76fe0f88ff1f 
Status: Connected 
Number of entries: 12 

Brick found3.ssd.org:/data/brick6/data 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45/82a7027b-321c-4bd9-8afc-2a12cfa23bfc 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45/82a7027b-321c-4bd9-8afc-2a12cfa23bfc.meta 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/d1164a9b-ba63-46c4-a9ec-76ea4a7a2c45 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/ce2831a4-3e82-4bf8-bb68-82afa0c401fe 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images - Is in split-brain 

/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/ee8e0fd3-cf68-4f2a-9d0d-f7ee665cc8c3 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/a4bea667-c65d-4085-9b90-896bf7fc55ff 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/4981580d-628f-4266-8e7e-f7a0bcae2dbb 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/2ee209b4-3e21-47fc-8342-033a37605d65 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/dom_md/xleases 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/dom_md 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images/28d32853-5c89-4c5f-8fb7-76fe0f88ff1f 
Status: Connected 
Number of entries: 12 

luster volume heal VMData2 info split-brain gives: 

Brick found2.ssd.org:/data/brick6/data 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images 
Status: Connected 
Number of entries in split-brain: 1 

Brick found3.ssd.org:/data/brick6/data 
/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images 
Status: Connected 
Number of entries in split-brain: 1 

on found3 running getfattr -d -m . hex /data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images gives: 

getfattr: hex: No such file or directory 
getfattr: data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images: No such file or directory 
[root at found3 ~]# getfattr -d -m . hex /data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images 
getfattr: hex: No such file or directory 
getfattr: Removing leading '/' from absolute path names 
# file: data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images 
security.selinux="system_u:object_r:default_t:s0" 
trusted.afr.VMData2-client-6=0sAAAAAQAAAAAAAAAG 
trusted.gfid=0sK8ZFxmThRxeq7pYw7QTOCw== 
trusted.glusterfs.dht=0sAAAAAQAAAABVVVVVqqqqqQ== 

on Found2 running getfattr -d -m . hex /data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images gives: 

getfattr: hex: No such file or directory 
getfattr: Removing leading '/' from absolute path names 
# file: data/brick6/data/08aa5fc4-c9ba-4fcf-af57-72450b875d1a/images 
security.selinux="system_u:object_r:default_t:s0" 
trusted.afr.VMData2-client-6=0sAAAAAQAAAAAAAAAG 
trusted.afr.dirty=0sAAAAAAAAAAAAAAAA 
trusted.gfid=0sK8ZFxmThRxeq7pYw7QTOCw== 
trusted.glusterfs.dht=0sAAAAAQAAAABVVVVVqqqqqQ== 

the only difference being the trusted.afr.dirty item in found2 which in not in found3. 

Any help would be appreciated. 




Russell Wecker 
IT Director 
Southern Asia-Pacific Division 
San Miguel II 
Bypass Silang, Silang, Cavite 
4118 Philippines 
Phone: +63 46 414 4000 x 5210 
Cell: +63 917 595 6395 
URL: [ http://ssd.adventist.asia/ | http://ssd.adventist.asia ] 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20180220/80a9844a/attachment.html>