<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:dt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:宋体;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@等线";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"\@宋体";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:宋体;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:宋体;}
p.m2020107212757358206msolistparagraph, li.m2020107212757358206msolistparagraph, div.m2020107212757358206msolistparagraph
{mso-style-name:m_2020107212757358206msolistparagraph;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:宋体;}
span.EmailStyle19
{mso-style-type:personal;
font-family:DengXian;
color:windowtext;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:DengXian;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="ZH-CN" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">In real world this also happened, and when this happened, I notice that the getfattr result of the remaining entry(from
gluster v heal info command) of sn-0 blame sn-2(</span><span lang="EN-US">trusted.afr.mstate-client-2=<span style="color:red">0x00000000000000000000001c)</span></span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> and sn-2 blame sn-0(</span><span lang="EN-US">trusted.afr.mstate-client-0=<span style="color:red">0x000000010000000000000003</span>)</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">,
sn-1 does not blame anyone. But the file within the remaining entry does not have any changelog blaming others on 3 sn nodes.!<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Since this issue just appear occasionally, So my test step is to simulate this situation in real world, I think
it is better that glusterfs could also consider this situation. Do you have any comments?</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Zhou, Cynthia (NSB - CN/Hangzhou)
<br>
<b>Sent:</b> Thursday, September 06, 2018 9:08 AM<br>
<b>To:</b> 'Pranith Kumar Karampuri' <pkarampu@redhat.com><br>
<b>Cc:</b> Gluster Devel <gluster-devel@gluster.org>; Ravishankar N <ravishankar@redhat.com><br>
<b>Subject:</b> RE: remaining entry in gluster volume heal info command even after reboot<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">This test steps is only to simulate the real test case(reboot+ftest), in real case it also happened occasionally.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Pranith Kumar Karampuri <<a href="mailto:pkarampu@redhat.com">pkarampu@redhat.com</a>>
<br>
<b>Sent:</b> Wednesday, September 05, 2018 5:32 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) <<a href="mailto:cynthia.zhou@nokia-sbell.com">cynthia.zhou@nokia-sbell.com</a>><br>
<b>Cc:</b> Gluster Devel <<a href="mailto:gluster-devel@gluster.org">gluster-devel@gluster.org</a>>; Ravishankar N <<a href="mailto:ravishankar@redhat.com">ravishankar@redhat.com</a>><br>
<b>Subject:</b> Re: remaining entry in gluster volume heal info command even after reboot<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US">Looks like the test case is a bit involved and also has modifications directly on the brick. Could you let us know if there is any reason to touch the brick directly?<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">On Wed, Sep 5, 2018 at 2:53 PM Zhou, Cynthia (NSB - CN/Hangzhou) <<a href="mailto:cynthia.zhou@nokia-sbell.com">cynthia.zhou@nokia-sbell.com</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">I will try to reproduce (reboot + ftest)and tell you later, but in following steps you can also simulate this issue
locally, at least the remaining entry happened and the entry heal quit also because of all thee empty heald_sinks.(not sure if this is exactly the same from reboot+ftest reproduced one but I guess it should be the same)</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">1></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Stop client quorum by command
</span><span style="font-size:10.5pt;font-family:DengXian">“<span lang="EN-US">gluster v set <volname> cluster.quorum-type none</span>”</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">2></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Isolate sn-0 from sn-1 and sn-2</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -I OUTPUT -d sn-1.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -I OUTPUT -d sn-2.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -I INPUT -s sn-2.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -I INPUT -s sn-1.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">3></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Touch /mnt/export/testdir/common.txt on sn-0</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">4></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Touch /mnt/export/testidir/common.txt on sn-1</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">5></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">On sn-1 node,Delete all /mnt/bricks/export/brick/testdir/common.txt metadata until getfattr returns empty,
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">setfattr -x trusted.afr.dirty /mnt/bricks/export/brick/testdir/common.txt
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">setfattr -x trusted.afr.export-client-0 /mnt/bricks/export/brick/testdir/common.txt</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">setfattr -x trusted.gfid /mnt/bricks/export/brick/testdir/common.txt</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">setfattr -x trusted.gfid2path.53be37be7f01389d /mnt/bricks/export/brick/testdir/common.txt</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">then getfattr returns empty</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">[root@sn-1:/home/robot]</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"># getfattr -m . -d -e hex /mnt/bricks/export/brick/testdir/common.txt</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">[root@sn-1:/home/robot]</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">6></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">then delete the corresponding entry(common.txt) in /mnt/bricks/export/brick/.glusterfs/indices/xattrop/</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">[root@sn-1:/home/robot]</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"># rm -rf /mnt/bricks/export/brick/.glusterfs/indices/xattrop/d0d237f7-0c43-4828-8720-dfb3792fe5fb</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">7>Restore network on sn-0 node.</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -D OUTPUT -d sn-1.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">
</span><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -D OUTPUT -d sn-2.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> iptables -D INPUT -s sn-1.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:16.5pt">
<span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">iptables -D INPUT -s sn-2.local -j DROP</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">7></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Do touch /mnt/export/testdir/common.txt on sn-0 node</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="m2020107212757358206msolistparagraph" style="margin-left:18.0pt"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">8></span><span lang="EN-US" style="font-size:7.0pt;font-family:"Times New Roman",serif">
</span><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Gluster v heal export info will show following and keep for long time</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"># gluster v heal export info</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Brick sn-0.local:/mnt/bricks/export/brick</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">/testdir
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Status: Connected</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Number of entries: 1</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Brick sn-1.local:/mnt/bricks/export/brick</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Status: Connected</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Number of entries: 0</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Brick sn-2.local:/mnt/bricks/export/brick</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">/testdir
</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Status: Connected</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian">Number of entries: 1</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:10.5pt;font-family:DengXian"> </span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Pranith
Kumar Karampuri <<a href="mailto:pkarampu@redhat.com" target="_blank">pkarampu@redhat.com</a>>
<br>
<b>Sent:</b> Wednesday, September 05, 2018 4:56 PM<br>
<b>To:</b> Zhou, Cynthia (NSB - CN/Hangzhou) <<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>><br>
<b>Cc:</b> Gluster Devel <<a href="mailto:gluster-devel@gluster.org" target="_blank">gluster-devel@gluster.org</a>>; Ravishankar N <<a href="mailto:ravishankar@redhat.com" target="_blank">ravishankar@redhat.com</a>><br>
<b>Subject:</b> Re: remaining entry in gluster volume heal info command even after reboot</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="EN-US"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">On Wed, Sep 5, 2018 at 1:27 PM Zhou, Cynthia (NSB - CN/Hangzhou) <<a href="mailto:cynthia.zhou@nokia-sbell.com" target="_blank">cynthia.zhou@nokia-sbell.com</a>>
wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Hi glusterfs experts:<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Good day!<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Recently when I do some test on my gluster env, I found that there are some remaining entries in command
</span>“<span lang="EN-US">gluster v heal mstate info</span>” <b><span lang="EN-US">even after reboot</span></b><span lang="EN-US">.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">
<span style="color:red">fstest_035ffc492ec43551a64087f9280ffe3e</span> is a folder in /mnt/mstate and in this folder only one file(<span style="color:red">fstest_458bb82d8884ed5c9dadec4ed93bec4e</span>) exists.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:21.0pt">
<span lang="EN-US">When I dbg by gdb on sn-0 I found that:<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:47.25pt">
<span lang="EN-US">Parent dir changelog(<span style="color:red">fstest_035ffc492ec43551a64087f9280ffe3e)</span> says need to heal entry, but the changelog/gfid/filetype of the only entry in parent dir shows there is nothing to be healed, so glustershd does
nothing every round of heal. And this entry will remain.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:47.25pt">
<span lang="EN-US">My gdb shows that each round of heal on sn-0 , it exits in function __afr_selfheal_entry (if (AFR_COUNT(healed_sinks, priv->child_count) == 0)), because in this case all three healed_sinks are zero.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">What is the return value of this function in gdb?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Have you any idea how to solve this issue from glusterfs pov?<br>
Thanks!<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[test steps]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Reboot three sn nodes( sn-0, sn-1, sn-2(arbiter)) sequentially, and on another node (with glusterfs clients) run fstest.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-indent:47.25pt">
<span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[problem description]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Remaining entries in
</span>“<span lang="EN-US">gluster v heal mstate info</span>”<span lang="EN-US"> command even after reboot sn-0 many times, the entries are still there!<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># gluster v heal mstate info<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick sn-0.local:/mnt/bricks/mstate/brick<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">/fstest_035ffc492ec43551a64087f9280ffe3e
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Status: Connected<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Number of entries: 1<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick sn-1.local:/mnt/bricks/mstate/brick<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Status: Connected<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Number of entries: 0<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick sn-2.local:/mnt/bricks/mstate/brick<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">/fstest_035ffc492ec43551a64087f9280ffe3e
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Status: Connected<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Number of entries: 1<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">////////////////////////////////////////////////////////////////////some env informations///////////////////////////////////////////////////////<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># gluster v info mstate<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Volume Name: mstate<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Type: Replicate<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Volume ID: 1d896674-17a2-4ae7-aa7c-c6e22013df99<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Status: Started<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Snapshot Count: 0<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Number of Bricks: 1 x (2 + 1) = 3<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Transport-type: tcp<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Bricks:<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick1: sn-0.local:/mnt/bricks/mstate/brick<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick2: sn-1.local:/mnt/bricks/mstate/brick<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Brick3: sn-2.local:/mnt/bricks/mstate/brick (arbiter)<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Options Reconfigured:<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">performance.client-io-threads: off<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">nfs.disable: on<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">transport.address-family: inet<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.server-quorum-type: none<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.quorum-reads: no<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.favorite-child-policy: mtime<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.consistent-metadata: on<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">network.ping-timeout: 42<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.quorum-type: auto<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">server.allow-insecure: on<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">cluster.server-quorum-ratio: 51%<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.dirty=0x000000000000000000000000<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-0=<span style="color:red">0x000000010000000000000003</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0xa0975560eaef4cb299467101de00446a<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.glusterfs.dht=0x000000010000000000000000ffffffff<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-0=<span style="color:red">0x000000000000000000000000</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0x9fc20f587f094182816390f056f7370f<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># cd /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># ls<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># stat fstest_458bb82d8884ed5c9dadec4ed93bec4e
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> File: fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Size: 0 Blocks: 8 IO Block: 4096 fifo<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Device: fd31h/64817d Inode: 22086 Links: 2<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Modify: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Change: 2018-08-30 04:33:17.553870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Birth: -<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-2:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># exit<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">logout<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Connection to sn-2.local closed.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.dirty=0x000000000000000000000000<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-2=<span style="color:red">0x00000000000000000000001c</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0xa0975560eaef4cb299467101de00446a<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.glusterfs.dht=0x000000010000000000000000ffffffff<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0x9fc20f587f094182816390f056f7370f<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># ls /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># stat /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> File: /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Size: 0 Blocks: 8 IO Block: 4096 fifo<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Device: fd31h/64817d Inode: 21899 Links: 2<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Modify: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Change: 2018-08-30 04:33:17.263809048 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Birth: -<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-0:/home/robot]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># ssh sn-1.local<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">USAGE OF THE ROOT ACCOUNT AND THE FULL BASH IS RECOMMENDED ONLY FOR LIMITED USE. PLEASE USE A NON-ROOT ACCOUNT AND THE SCLI SHELL (fsclish) AND/OR LIMITED BASH
SHELL.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Read /opt/nokia/share/security/readme_root.txt for more details.<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.dirty=0x000000000000000000000000<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-0=<span style="color:red">0x000000000000000000000000</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-2=<span style="color:red">0x000000000000000000000000</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0xa0975560eaef4cb299467101de00446a<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.glusterfs.dht=0x000000010000000000000000ffffffff<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># getfattr -m . -d -e hex /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">getfattr: Removing leading '/' from absolute path names<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># file: mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.afr.mstate-client-0=<span style="color:red">0x000000000000000000000000</span><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid=0x9fc20f587f094182816390f056f7370f<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">trusted.gfid2path.864159d77373ad5f=0x61303937353536302d656165662d346362322d393934362d3731303164653030343436612f6673746573745f3435386262383264383838346564356339646164656334656439336265633465<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># ls /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"># stat /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e
<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> File: /mnt/bricks/mstate/brick/fstest_035ffc492ec43551a64087f9280ffe3e/fstest_458bb82d8884ed5c9dadec4ed93bec4e<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> Size: 0 Blocks: 8 IO Block: 4096 fifo<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Device: fd31h/64817d Inode: 22168 Links: 2<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: (0644/prw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Access: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Modify: 2018-08-30 04:33:17.552870661 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Change: 2018-08-30 04:33:17.037673648 +0300<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Birth: -<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">[root@sn-1:/root]<o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> <o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><br clear="all">
<br>
-- <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Pranith<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-US"><br clear="all">
<br>
-- <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Pranith<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>