<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Thanks
<a href="mailto:aravinda@kadalu.tech" id="OWAAM60602" class="tWKOu mention ms-bgc-nlr ms-fcl-b" data-loopstyle="linkonly">
@Aravinda</a>, Please check the output of 'getfattr' command.</span></div>
<div class="elementToProof"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div class="elementToProof"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">[root@master1 .glusterfs]# getfattr -d -m . ./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"># file: d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">trusted.afr.dirty=0sAAAAAAAAAAAAAAAA</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">trusted.gfid=0s1T+tj4TpSySesMy8vcS6qA==</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">trusted.gfid2path.5007d273788799dd="e8aff729-a310-4d21-a64b-d8cc7cb1a828/app_docmerge12monthsfixedCUSTODIAL_2024_1_22_15_3_24_648.doc"</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">trusted.glusterfs.93c45c14-f700-4d50-962b-7653be471e27.xtime=0sZa6EgQAFa7I=</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">trusted.glusterfs.mdata=0sAQAAAAAAAAAAAAAAAGWuhIEAAAAADGLMhwAAAABlroSBAAAAAAxizIcAAAAAZa6EgAAAAAAq348C</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">I will try to move the file and start geo-replication in an hour and let you know how it goes. I am also searching
for the change logs as you mentioned, but it may take time to grep.</span></div>
<div><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">I truly appreciate everyone's input so far.</span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Kind regards,</span></p>
<p style="margin-top: 0px; margin-bottom: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Anant</span></p>
<hr style="display: inline-block; width: 98%;">
<div dir="ltr" id="divRplyFwdMsg"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> Aravinda <aravinda@kadalu.tech><br>
<b>Sent:</b> 09 February 2024 4:39 PM<br>
<b>To:</b> Anant Saraswat <anant.saraswat@techblue.co.uk><br>
<b>Cc:</b> Diego Zuccato <diego.zuccato@unibo.it>; gluster-users@gluster.org <gluster-users@gluster.org>; Strahil Nikolov <hunter86_bg@yahoo.com><br>
<b>Subject:</b> Re: [Gluster-users] __Geo-replication status is getting Faulty after few seconds</span>
<div> </div>
</div>
<p><span style="background-color: rgb(250, 247, 190);"><b>EXTERNAL: Do not click links or open attachments if you do not recognize the sender.</b></span></p>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">If the backend GFID file only exists in Master 1, then it looks like stale GFID exists due to some failure. Run getfattr on that file to see if it shows any file path in
gfid2path xattr. Please share the output of the following command.</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">getfattr -d -m . ./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">Meanwhile move this file to a backup location and see if Geo-rep succeeds.</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">We can introspect the details about the GFID d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8. Please share the matching changelog files from </span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">grep -R "d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8" .glusterfs/changelogs/*</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">--</span></div>
<div data-zbluepencil-ignore="true" data-sigid="3848334000000010003" id="x_Zm-_Id_-Sgn">
<div>Aravinda</div>
</div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">Kadalu Technologies</span></div>
<div style="line-height: 0px; margin-top: 10px; margin-bottom: 10px; border-top: 1px solid rgb(204, 204, 204); height: 0px;">
<span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div id="x_Zm-_Id_-Sgn1">---- On Fri, 09 Feb 2024 16:23:45 +0530 <b>Anant Saraswat <anant.saraswat@techblue.co.uk></b> wrote ---<br>
</div>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi <a href="mailto:hunter86_bg@yahoo.com" id="OWA1fddd4ee-836d-2827-d9cd-3164290f6594" class="x_x_1748230667tWKOu x_x_1748230667mention x_x_1748230667ms-bgc-nlr x_x_1748230667ms-fcl-b OWAAutoLink" data-loopstyle="linkonly">
@Strahil Nikolov</a>,</div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">As mentioned earlier this gfid has "<b>1</b>"
in 2<sup>nd</sup> column of 'ls' command output, which means this gfid doesn't have any other hardlink.</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><i>[root@master1 .glusterfs]# ls -l ./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8<br>
-rw-r--r-- </i><b><i>1 </i></b><i>root root 174592 Jan 22 15:06 ./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8</i></span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Still, I have searched in "/opt/tier1data2019/brick/.glusterfs"
and there is only one file comes with samefile option, which is gfid itself, please check the following output.</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><i>[root@master1 .glusterfs]# find . -samefile
./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8<br>
./d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8</i></span></div>
<div id="x_x_1748230667appendonsend"><br>
</div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">You have mentioned the other option to link it
somewhere on the real filesystem and then sync it to the other brick. Here, what do you mean by the real filesystem? Do you want me to link this GFID with an existing file on the brick? Or do you want the GFID to be linked with some temporary file, say, on
"/tmp/test_file," and then sync "/tmp/test_file" and "/opt/tier1data2019/brick/.glusterfs/d5/3f/d53fad8f-84e9-4b24-9eb0-ccbcbdc4baa8" to master2 and master3 nodes?</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Many thanks,</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Anant</span></div>
<hr style="direction: ltr; margin-right: 0px; margin-left: 0px; display: inline-block; width: 98%;">
<div id="x_x_1748230667divRplyFwdMsg" dir="ltr"><span style="font-family: Calibri, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);"><b>From:</b> Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com" id="OWA6944d8c8-9ed2-7f4a-717b-b0e00cb1ca67" class="OWAAutoLink" data-loopstyle="linkonly">hunter86_bg@yahoo.com</a>><br>
<b>Sent:</b> 09 February 2024 5:28 AM<br>
<b>To:</b> Anant Saraswat <<a href="mailto:anant.saraswat@techblue.co.uk" id="OWA8fb93ad2-1f63-2d03-76ba-2aed3d834cad" class="OWAAutoLink" data-loopstyle="linkonly">anant.saraswat@techblue.co.uk</a>>; Diego Zuccato <<a href="mailto:diego.zuccato@unibo.it" id="OWAfdc86e50-a2cc-70af-9a94-e38d24bb5185" class="OWAAutoLink" data-loopstyle="linkonly">diego.zuccato@unibo.it</a>>;
<a href="mailto:gluster-users@gluster.org" id="OWAe67dbe7e-78ce-f99a-f531-41c10a67a2d1" class="OWAAutoLink" data-loopstyle="linkonly">
gluster-users@gluster.org</a> <<a href="mailto:gluster-users@gluster.org" id="OWAb0915899-4502-8733-dcaf-06726e5e350d" class="OWAAutoLink" data-loopstyle="linkonly">gluster-users@gluster.org</a>>; Aravinda Vishwanathapura <<a href="mailto:aravinda@kadalu.tech" id="OWAc96c0d28-8c6e-0034-7320-90767a35243b" class="OWAAutoLink" data-loopstyle="linkonly">aravinda@kadalu.tech</a>><br>
<b>Subject:</b> Re: [Gluster-users] __Geo-replication status is getting Faulty after few seconds</span>
<div> </div>
</div>
<p style="direction: ltr; margin: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt; background-color: rgb(250, 247, 190);"><b>EXTERNAL: Do not click links or open attachments if you do not recognize the sender.</b></span></p>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">The other option around (if indeed is the only in .glusterfs) is to have it link somewhere on the real filesystem
structure and then sync it to the other bricks.</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">If georep works again - just remove it from the fuse and check if it will get deleted or not.</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">Best Regards,</span></div>
<div style="direction: ltr; margin-right: 0px; margin-left: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;">Strahil Nikolov<br>
<br>
</span></div>
<blockquote style="margin: 0px 0px 20px;">
<div style="direction: ltr;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; color: rgb(109, 0, 246);">On Fri, Feb 9, 2024 at 7:25, Strahil Nikolov</span></div>
<div style="direction: ltr;"><span style="font-family: Roboto, sans-serif; font-size: 10pt; color: rgb(109, 0, 246);"><<a href="mailto:hunter86_bg@yahoo.com" id="OWA1c9c0be9-645f-d3a5-dbbf-d9a0e1936500" class="OWAAutoLink" data-loopstyle="linkonly">hunter86_bg@yahoo.com</a>>
wrote:</span></div>
<div style="margin: 10px 0px 0px; padding-top: 10px; padding-left: 20px; border-left: 1px solid rgb(109, 0, 246);">
<div id="x_x_yiv3593417850">
<div>It's a hard link, so use find's '-samefile' option to see if it's the last one or not.</div>
<div><br>
</div>
<div>If you really want to delete it, have a backup and then delete both the gfid and any other hard links.</div>
<div><br>
</div>
<div>Best Regards,</div>
<div>Strahil Nikolov<br>
<br>
</div>
<div id="x_x_yiv3593417850yqt91580" class="x_x_yiv3593417850yqt7501725214"><br>
</div>
</div>
</div>
</blockquote>
<p style="direction: ltr; margin: 0px;"><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></p>
<p style="direction: ltr; margin: 0px;"><span style="font-family: ARIAL; font-size: 10pt;">DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you
have received this email in error, please notify the sender. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please
notify the sender immediately by email if you have received this email by mistake and delete this email from your system.<br>
<br>
If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Thanks for your cooperation.</span></p>
<div><span style="font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 10pt;"><br>
</span></div>
<br>
<p style="FONT-SIZE: 10pt; FONT-FAMILY: ARIAL"><span style="FONT-FAMILY: Calibri Light"></p>
<p style="FONT-SIZE: 10pt; FONT-FAMILY: ARIAL">DISCLAIMER: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error, please
notify the sender. This message contains confidential information and is intended only for the individual named. If you are not the named addressee, you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if
you have received this email by mistake and delete this email from your system. <br>
<br>
If you are not the intended recipient, you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited. Thanks for your cooperation.
</span></p>
</body>
</html>