<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office"><head><!--[if gte mso 9]><xml><o:OfficeDocumentSettings><o:AllowPNG/><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml><![endif]--></head><body><div class="ydp4f1d253eyahoo-style-wrap" style="font-family: courier new, courier, monaco, monospace, sans-serif; font-size: 16px;"><div></div>
<div dir="ltr" data-setdir="false">It seems that whenever I reboot a gluster node , I got this problem - so it's not an arbiter issue.</div><div dir="ltr" data-setdir="false">Obviously there is something wrong with v6.6 ,as I never had such issues with v6.5 .</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Any ideas where should I start this up ?</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Best Regards,</div><div dir="ltr" data-setdir="false">Strahil Nikolov</div><div><br></div>
</div><div id="ydp4c23ee86yahoo_quoted_4660572221" class="ydp4c23ee86yahoo_quoted">
<div style="font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;font-size:13px;color:#26282a;">
<div>
В сряда, 13 ноември 2019 г., 22:23:38 ч. Гринуич+2, Strahil Nikolov <hunter86_bg@yahoo.com> написа:
</div>
<div><br></div>
<div><br></div>
<div><div id="ydp4c23ee86yiv5937751135"><!--[if gte mso 9]><xml><o:OfficeDocumentSettings><o:AllowPNG/><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml><![endif]--><div><div class="ydp4c23ee86yiv5937751135yahoo-style-wrap" style="font-family:courier new, courier, monaco, monospace, sans-serif;font-size:16px;"><div dir="ltr">Hello Community,</div><div dir="ltr"><br></div><div dir="ltr">I had a power issue last night and all the systems went down. When they powered up the arbiter was offline (network issue) which was resolved and heal passed for almost all files but 2 per volume.</div><div dir="ltr"><br></div><div dir="ltr">The strange thing is that the arbiter's copy is older and the '<span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">trusted.gfid' is different on the arbiter. In which cases this can happen ?</span><br></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><br></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">I guess the heal is not passing ,as the 'trusted.gfid' is different while '<span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">trusted.gfid2path.<long_string>' is the same.</span><br></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">Here is the output of getfattr and I need to realise if this is a possible situation or it is a bug that needs reporting:</span></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">Brick1:</span></span></span></span></span></span></div><div dir="ltr"><span><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);"><span><span style="font-family:monospace;"><div><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">[root@ovirt1 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.data_fast-client-2=0x000000020000000200000000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x2408e73df137431e99c663783b8bb356
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcc1eae0000000023c985d9000000005dcc1eae000000001f4c18bf000000005dcc1eae000000001ed22d12
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000<br></span></div><div><span style="font-family:monospace;"><br></span></div><div><span style="font-family:monospace;"><br></span></div><div dir="ltr"><span style="font-family:monospace;">Brick2:</span></div><div dir="ltr"><div><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">[root@ovirt2 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.data_fast-client-2=0x000000020000000200000000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x2408e73df137431e99c663783b8bb356
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcc1eae0000000023c985d9000000005dcc1eae000000001f4c18bf000000005dcc1eae000000001ed22d12
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000
<br><br>
<br></span></div>Arbiter:</div><div dir="ltr"><div><span style="font-family:monospace;"><span style="color:rgb(0, 0, 0);">[root@ovirt3 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x48c9c4ebe75c41f7ac45ea4943e9a65a
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcb3a520000000011c7ae2c000000005dcb3a520000000011690f7b000000005dcb3a520000000011690f7b
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000<br></span></div><br></div><div dir="ltr">Also, as per '<span><a href="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" class="ydp4c23ee86yiv5937751135enhancr_card_3709696368" rel="nofollow" target="_blank">Troubleshooting Split-Brains - Gluster Docs</a>' there should be another 2 copies of the changelog (local + remote bricks) but in my case they are missing. Why is it like that ?</span></div><div dir="ltr"><br></div><div dir="ltr"><span><span style="font-family:Roboto, Helvetica, Arial, sans-serif;font-size:16px;">"Each file in a brick maintains the changelog of itself and that of the files present in all the other bricks in its replica set as seen by that brick."</span></span><br><div><br></div><div id="ydp4c23ee86yiv5937751135ydp429242e9enhancr_card_3709696368" class="ydp4c23ee86yiv5937751135ydp429242e9yahoo-link-enhancr-card ydp4c23ee86yiv5937751135ydp429242e9yahoo-link-enhancr-not-allow-cover ydp4c23ee86yiv5937751135ydp429242e9ymail-preserve-class ydp4c23ee86yiv5937751135ydp429242e9ymail-preserve-style" style="max-width:400px;font-family:Helvetica, Arial, sans-serif;" data-url="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" data-type="YENHANCER" data-size="MEDIUM"><a href="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" style="text-decoration:none !important;color:#000 !important;" class="ydp4c23ee86yiv5937751135ydp429242e9yahoo-enhancr-cardlink" rel="nofollow" target="_blank"><table border="0" class="ydp4c23ee86yiv5937751135ydp429242e9card-wrapper ydp4c23ee86yiv5937751135ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" style="max-width:400px;"><tbody><tr><td width="400"><table border="0" class="ydp4c23ee86yiv5937751135ydp429242e9card ydp4c23ee86yiv5937751135ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" width="100%" style="max-width:400px;border-width:1px;border-style:solid;border-color:rgb(224, 228, 233);border-radius:2px;"><tbody><tr><td><table border="0" class="ydp4c23ee86yiv5937751135ydp429242e9card-info ydp4c23ee86yiv5937751135ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" style="background:#fff;position:relative;z-index:2;width:100%;max-width:400px;border-radius:0 0 2px 2px;border-top:1px solid rgb(224, 228, 233);"><tbody><tr><td style="background-color:#ffffff;padding:16px 0 16px 12px;vertical-align:top;border-radius:0 0 0 2px;"></td><td style="vertical-align:middle;padding:12px 24px 16px 12px;width:99%;font-family:Helvetica, Arial, sans-serif;border-radius:0 0 2px 0;"><h2 class="ydp4c23ee86yiv5937751135ydp429242e9card-title" style="font-size:14px;line-height:19px;margin:0px 0px 6px;font-family:Helvetica, Arial, sans-serif;color:rgb(38, 40, 42);">Troubleshooting Split-Brains - Gluster Docs</h2><p class="ydp4c23ee86yiv5937751135ydp429242e9card-description" style="font-size:12px;line-height:16px;margin:0px;color:rgb(151, 155, 167);">Project documentation for Gluster Filesystem</p></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></a></div><div dir="ltr"><br></div><div dir="ltr">Thanks in advance for your response.</div><div dir="ltr"><br></div><div dir="ltr">Best Regards,</div><div dir="ltr">Strahil Nikolov</div><div><br></div></div></span></span></span></span></span></div></div></div></div></div>
</div>
</div></body></html>