<html xmlns="http://www.w3.org/1999/xhtml" xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office"><head><!--[if gte mso 9]><xml><o:OfficeDocumentSettings><o:AllowPNG/><o:PixelsPerInch>96</o:PixelsPerInch></o:OfficeDocumentSettings></xml><![endif]--></head><body><div class="yahoo-style-wrap" style="font-family:courier new, courier, monaco, monospace, sans-serif;font-size:16px;"><div dir="ltr" data-setdir="false">Hello Community,</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">I had a power issue last night and all the systems went down. When they powered up the arbiter was offline (network issue) which was resolved and heal passed for almost all files but 2 per volume.</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">The strange thing is that the arbiter's copy is older and the '<span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">trusted.gfid' is different on the arbiter. In which cases this can happen ?</span><br></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><br></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">I guess the heal is not passing ,as the 'trusted.gfid' is different while '<span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">trusted.gfid2path.<long_string>' is the same.</span><br></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">Here is the output of getfattr and I need to realise if this is a possible situation or it is a bug that needs reporting:</span></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><br></span></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">Brick1:</span></span></span></span></span></span></div><div dir="ltr" data-setdir="false"><span><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);"><span><span style="font-family:monospace"><div><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">[root@ovirt1 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.data_fast-client-2=0x000000020000000200000000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x2408e73df137431e99c663783b8bb356
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcc1eae0000000023c985d9000000005dcc1eae000000001f4c18bf000000005dcc1eae000000001ed22d12
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000<br></span></div><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace"><br></span></div><div dir="ltr" data-setdir="false"><span style="font-family:monospace">Brick2:</span></div><div dir="ltr" data-setdir="false"><div><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">[root@ovirt2 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.data_fast-client-2=0x000000020000000200000000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x2408e73df137431e99c663783b8bb356
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcc1eae0000000023c985d9000000005dcc1eae000000001f4c18bf000000005dcc1eae000000001ed22d12
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000
<br><br>
<br></span></div>Arbiter:</div><div dir="ltr" data-setdir="false"><div><span style="font-family:monospace"><span style="color: rgb(0, 0, 0);">[root@ovirt3 ~]# getfattr -d -m . -e hex /gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673b</span><br>b2.meta
<br>getfattr: Removing leading '/' from absolute path names
<br># file: gluster_bricks/data_fast/data_fast/396604d9-2a9e-49cd-9563-fdc79981f67b/images/25107759-f56b-4d94-872e-0f9b145886d1/571a5eb4-eb92-479c-8305-79eaa0673bb2.meta
<br>security.selinux=0x73797374656d5f753a6f626a6563745f723a756e6c6162656c65645f743a733000
<br>trusted.afr.dirty=0x000000000000000000000000
<br>trusted.gfid=0x48c9c4ebe75c41f7ac45ea4943e9a65a
<br>trusted.gfid2path.27d407743ed6191b=0x35333430653763322d666333302d343461382d613633392d3762613138636138653033362f35373161356562342d656239322d343739632d383330352d3739656161303637336262322e6d65746<br>1
<br>trusted.glusterfs.mdata=0x010000000000000000000000005dcb3a520000000011c7ae2c000000005dcb3a520000000011690f7b000000005dcb3a520000000011690f7b
<br>trusted.glusterfs.shard.block-size=0x0000000004000000
<br>trusted.glusterfs.shard.file-size=0x00000000000001b4000000000000000000000000000000010000000000000000<br></span></div><br></div><div dir="ltr" data-setdir="false">Also, as per '<span><a href="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" rel="nofollow" target="_blank" class="enhancr_card_3709696368">Troubleshooting Split-Brains - Gluster Docs</a>' there should be another 2 copies of the changelog (local + remote bricks) but in my case they are missing. Why is it like that ?</span></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false"><span><span style="font-family: Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 16px;">"Each file in a brick maintains the changelog of itself and that of the files present in all the other bricks in its replica set as seen by that brick."</span></span><br><div><br></div><div id="ydp429242e9enhancr_card_3709696368" class="ydp429242e9yahoo-link-enhancr-card ydp429242e9yahoo-link-enhancr-not-allow-cover ydp429242e9ymail-preserve-class ydp429242e9ymail-preserve-style" style="max-width:400px;font-family:"Helvetica Neue", "Segoe UI", Helvetica, Arial, sans-serif" data-url="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" data-type="YENHANCER" data-size="MEDIUM" contenteditable="false"><a href="https://docs.gluster.org/en/latest/Troubleshooting/resolving-splitbrain/" style="text-decoration:none !important;color:#000 !important" class="ydp429242e9yahoo-enhancr-cardlink" rel="nofollow" target="_blank"><table border="0" class="ydp429242e9card-wrapper ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" style="max-width:400px"><tbody><tr><td width="400"><table border="0" class="ydp429242e9card ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" width="100%" style="max-width:400px;border-width:1px;border-style:solid;border-color:rgb(224, 228, 233);border-radius:2px"><tbody><tr><td><table border="0" class="ydp429242e9card-info ydp429242e9yahoo-ignore-table" cellpadding="0" cellspacing="0" style="background:#fff;position:relative;z-index:2;width:100%;max-width:400px;border-radius:0 0 2px 2px;border-top:1px solid rgb(224, 228, 233)"><tbody><tr><td style="background-color:#ffffff;padding:16px 0 16px 12px;vertical-align:top;border-radius:0 0 0 2px"></td><td style="vertical-align:middle;padding:12px 24px 16px 12px;width:99%;font-family:"Helvetica Neue", "Segoe UI", Helvetica, Arial, sans-serif;border-radius:0 0 2px 0"><h2 class="ydp429242e9card-title" style="font-size: 14px; line-height: 19px; margin: 0px 0px 6px; font-family: "Helvetica Neue", "Segoe UI", Helvetica, Arial, sans-serif; color: rgb(38, 40, 42);">Troubleshooting Split-Brains - Gluster Docs</h2><p class="ydp429242e9card-description" style="font-size: 12px; line-height: 16px; margin: 0px; color: rgb(151, 155, 167);">Project documentation for Gluster Filesystem</p></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table></a></div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Thanks in advance for your response.</div><div dir="ltr" data-setdir="false"><br></div><div dir="ltr" data-setdir="false">Best Regards,</div><div dir="ltr" data-setdir="false">Strahil Nikolov</div><div><br></div></div></span></span></span></span></span></div></div></body></html>