<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from rtf -->
<style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<font face="Times New Roman" size="2"><span style="font-size:10.5pt;">
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Calibri">HI gluster experts,</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">I meet a tough problem about ¡°split-brain¡± issue. Sometimes, after hard reboot, we will find some files in split-brain, however its parent directory or anything could be shown in command ¡°gluster
volume heal <volume-name> info¡±, also, no entry in .glusterfs/indices/xattrop directory, can you help to shed some lights on this issue? Thanks!</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Following is some info from our env,</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"><b>Checking from sn-0 cliet, nothing is shown in-split-brain!</b></font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># gluster v heal services info</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Brick sn-0:/mnt/bricks/services/brick/</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Number of entries: 0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Brick sn-1:/mnt/bricks/services/brick/</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Number of entries: 0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># gluster v heal services info split-brain</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Gathering list of split brain entries on volume services has been successful </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Brick sn-0.local:/mnt/bricks/services/brick</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Number of entries: 0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Brick sn-1.local:/mnt/bricks/services/brick</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Number of entries: 0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># ls -l /mnt/services/netserv/ethip/</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri" color="red">ls: cannot access '/mnt/services/netserv/ethip/sn-2': Input/output error</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri" color="red">ls: cannot access '/mnt/services/netserv/ethip/mn-1': Input/output error</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">total 3</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 144 Sep 26 20:35 as-0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 144 Sep 26 20:35 as-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 145 Sep 26 20:35 as-2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 237 Sep 26 20:36 mn-0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-????????? ? ? ? ? ? mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 73 Sep 26 20:35 sn-0</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-rw-r--r-- 1 root root 73 Sep 26 20:35 sn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">-????????? ? ? ? ? ? sn-2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"><b>Checking from glusterfs server side, the gfid of mn-1 on sn-0 and sn-1 is different</b></font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"><b>[SN-0]</b></font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/.glusterfs/53/a3]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># getfattr -m . -d -e hex /mnt/bricks/services/brick/netserv/ethip</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">getfattr: Removing leading '/' from absolute path names</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># file: mnt/bricks/services/brick/netserv/ethip</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.gfid=0xee71d19ac0f84f60b11eb42a083644e4</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.glusterfs.dht=0x000000010000000000000000ffffffff</font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># getfattr -m . -d -e hex mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># file: mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.dirty=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.services-client-0=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.services-client-1=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.gfid=<font color="red">0x53a33f437464475486f31c4e44d83afd</font></font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># stat mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> File: mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Size: 237 Blocks: 16 IO Block: 4096 regular file</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Device: fd51h/64849d Inode: 2536 Links: 2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: 2017-09-26 20:30:25.679000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Modify: 2017-09-26 20:30:24.604000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Change: 2017-09-26 20:30:24.610000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Birth: -</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/.glusterfs/indices/xattrop]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># ls</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">xattrop-63f8bbcb-7fa6-4fc8-b721-675a05de0ab3</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/.glusterfs/indices/xattrop]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/.glusterfs/53/a3]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># ls</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">53a33f43-7464-4754-86f3-1c4e44d83afd</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-0:/mnt/bricks/services/brick/.glusterfs/53/a3]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># stat 53a33f43-7464-4754-86f3-1c4e44d83afd </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> File: 53a33f43-7464-4754-86f3-1c4e44d83afd</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Size: 237 Blocks: 16 IO Block: 4096 regular file</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Device: fd51h/64849d Inode: 2536 Links: 2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: 2017-09-26 20:30:25.679000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Modify: 2017-09-26 20:30:24.604000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Change: 2017-09-26 20:30:24.610000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Birth: -</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">#</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"><b>[SN-1]</b></font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/.glusterfs/f7/f1]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># getfattr -m . -d -e hex /mnt/bricks/services/brick/netserv/ethip</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">getfattr: Removing leading '/' from absolute path names</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># file: mnt/bricks/services/brick/netserv/ethip</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.gfid=0xee71d19ac0f84f60b11eb42a083644e4</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.glusterfs.dht=0x000000010000000000000000ffffffff</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/.glusterfs/f7/f1]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"><b>#</b></font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># getfattr -m . -d -e hex mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># file: mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.dirty=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.services-client-0=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.afr.services-client-1=0x000000000000000000000000</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">trusted.gfid=<font color="red">0xf7f10f980acc4041a015e48018571d4a</font></font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/netserv/ethip]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># stat mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> File: mn-1</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Size: 237 Blocks: 16 IO Block: 4096 regular file</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Device: fd41h/64833d Inode: 2608 Links: 2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: 2017-09-26 20:31:48.231000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Modify: 2017-09-26 20:31:46.872000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Change: 2017-09-26 20:31:46.875000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Birth: -</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/.glusterfs/indices/xattrop]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># ls</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">xattrop-240713ea-eda3-4914-a55d-7dd4aed724ed</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/.glusterfs/indices/xattrop]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">[root@sn-1:/mnt/bricks/services/brick/.glusterfs/f7/f1]</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"># stat f7f10f98-0acc-4041-a015-e48018571d4a </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> File: f7f10f98-0acc-4041-a015-e48018571d4a</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Size: 237 Blocks: 16 IO Block: 4096 regular file</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Device: fd41h/64833d Inode: 2608 Links: 2</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Access: 2017-09-26 20:31:48.231000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Modify: 2017-09-26 20:31:46.872000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri">Change: 2017-09-26 20:31:46.875000000 +0300</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> Birth: -</font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Calibri"> </font></div>
<div align="left" style="text-align:justify;"><font face="Arial">Best regards,<br>
<b>Cynthia </b><font face="ËÎÌå"><b>£¨ÖÜÁÕ£©</b></font></font></div>
<div align="left" style="text-align:justify;"><font face="Arial">MBB SM HETRAN SW3 MATRIX </font></div>
<div align="left" style="text-align:justify;"><font face="Arial">Storage <br>
Mobile: +86 (0)18657188311</font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"><font face="Arial">Best regards,<br>
<b>Cynthia </b><font face="ËÎÌå"><b>£¨ÖÜÁÕ£©</b></font></font></div>
<div align="left" style="text-align:justify;"><font face="Arial">MBB SM HETRAN SW3 MATRIX </font></div>
<div align="left" style="text-align:justify;"><font face="Arial">Storage <br>
Mobile: +86 (0)18657188311</font></div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"> </div>
<div align="left" style="text-align:justify;"> </div>
</span></font>
</body>
</html>