<div dir="ltr">Just to confirm I've got this correct?<div><br><div>So I'll move the directory with the different gfid on the Arbiter brick to somewhere else</div><div>I then touch this directory on another brick(software is not sensitive to atime update)</div></div><div><br></div><div>I guess the healing should then take place automatically?</div><div><br></div><div>Thanks</div><div>David</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 23 Feb 2023 at 11:01, Strahil Nikolov <<a href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Move away the file located onthe arbiter brick as it has different gfid and touch it(only if the software that consumes it is NOT sensitive to atime modification).<div><div><br></div><div>Best Regards,</div><div>Strahil Nikolov </div><div> <br> <blockquote style="margin:0px 0px 20px"> <div style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Wed, Feb 22, 2023 at 13:09, David Dolan</div><div><<a href="mailto:daithidolan@gmail.com" target="_blank">daithidolan@gmail.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246)"> <div id="m_-3239097665576610503yiv9993888487"><div><div dir="ltr">Hi Strahil,<div><br clear="none"></div><div>The output in my previous email showed the directory the file is located in with a different GFID on the Arbiter node compared with the bricks on the other nodes.</div><div><br clear="none"></div><div>Based on that, do you know what my next step should be?</div><div><br clear="none"></div><div>Thanks</div><div>David</div><div><br clear="none"></div></div><br clear="none"><div id="m_-3239097665576610503yiv9993888487yqt02617"><div><div dir="ltr">On Wed, 15 Feb 2023 at 09:21, David Dolan <<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:daithidolan@gmail.com" target="_blank">daithidolan@gmail.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">sorry I didn't receive the previous email.<div>I've run the command on all 3 nodes(bricks). See below. The directory only has one file.</div><div>On the Arbiter, the file doesn't exist and the directory the file should be in has a different GFID than the bricks on the other nodes</div><div><br clear="none"></div><div>Node 1 Brick</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2/file</div><div>trusted.gfid=0x7b1aa40dd1e64b7b8aac7fc6bcbc9e9b<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2<br clear="none"></div><div>trusted.gfid=0xdc99ac0db85d4b1c8a6af57a71bbe22c<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Node 2 Brick</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2/file<br clear="none"></div><div>trusted.gfid=0x7b1aa40dd1e64b7b8aac7fc6bcbc9e9b<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2</div><div>trusted.gfid=0xdc99ac0db85d4b1c8a6af57a71bbe22c<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Node 3 Brick (Arbiter)</div><div>Path to file doesn't exist</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2<br clear="none"></div><div>trusted.gfid=0x51cca97ac2974ceb9322fe21e6f8ea91<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Thanks</div><div>David</div></div><br clear="none"><div><div dir="ltr">On Tue, 14 Feb 2023 at 20:38, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:hunter86_bg@yahoo.com" target="_blank">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div> I guess you didn't receive my last e-mail.<br clear="none">Use getfattr and identify if the gfid mismatch. If yes, move away the mismatched one.<br clear="none">In order a dir to heal, you have to fix all files inside it before it can be healed.<br clear="none"><br clear="none">Best Regards,<br clear="none">Strahil Nikolov </div> <div style="margin:10px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div style="font-family:Helvetica,Arial,sans-serif;font-size:13px;color:rgb(38,40,42)"> <div> В вторник, 14 февруари 2023 г., 14:04:31 ч. Гринуич+2, David Dolan <<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:daithidolan@gmail.com" target="_blank">daithidolan@gmail.com</a>> написа: </div> <div><br clear="none"></div> <div><br clear="none"></div> <div><div id="m_-3239097665576610503yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711"><div><div dir="ltr">I've touched the directory one level above the directory with the I\O issue as the one above that is the one showing as dirty.<div>It hasn't healed. Should the self heal daemon automatically kick in here?</div><div><br clear="none"></div><div>Is there anything else I can do?</div><div><br clear="none"></div><div>Thanks</div><div>David</div></div><br clear="none"><div id="m_-3239097665576610503yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711yqt33930"><div><div dir="ltr">On Tue, 14 Feb 2023 at 07:03, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:hunter86_bg@yahoo.com" target="_blank">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">You can always mount it locally on any of the gluster nodes.<div><br clear="none"></div><div>Best Regards,</div><div>Strahil Nikolov <br clear="none"> <br clear="none"> <blockquote style="margin:0px 0px 20px"> <div style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Mon, Feb 13, 2023 at 18:13, David Dolan</div><div><<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:daithidolan@gmail.com" target="_blank">daithidolan@gmail.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246)"> <div id="m_-3239097665576610503yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743"><div><div dir="ltr"><div dir="ltr">HI Strahil,<div><br clear="none"></div><div>Thanks for that. It's the first time I've been in this position, so I'm learning as I go along.</div><div><br clear="none"></div><div>Unfortunately I can't go into the directory on the client side as I get an input/output error</div><div>Input/output error<br clear="none"></div><div>d????????? ? ? ? ? ? 01<br clear="none"></div><div><br clear="none"></div><div>Thanks</div><div>David</div><div><br clear="none"></div></div><br clear="none"><div id="m_-3239097665576610503yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743yqt14550"><div><div dir="ltr">On Sun, 12 Feb 2023 at 20:29, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:hunter86_bg@yahoo.com" target="_blank">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Setting blame on client-1 and client-2 will make a bigger mess.<div>Can't you touch the affected file from the FUSE mount point ?</div><div><br clear="none"></div><div>Best Regards,</div><div>Strahil Nikolov <br clear="none"> <br clear="none"> <blockquote style="margin:0px 0px 20px"> <div style="font-family:Roboto,sans-serif;color:rgb(109,0,246)"> <div>On Tue, Feb 7, 2023 at 14:42, David Dolan</div><div><<a rel="nofollow noopener noreferrer" shape="rect" href="mailto:daithidolan@gmail.com" target="_blank">daithidolan@gmail.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246)"> <div id="m_-3239097665576610503yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743m_4113625973783826226yiv8260473842"><div dir="ltr"><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)">Hi All. </div><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)"><br clear="none"></div><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)">Hoping you can help me with a healing problem. I have one file which didn't self heal.<br clear="none">it looks to be a problem with a directory in the path as one node says it's dirty. I have a replica volume with arbiter<br clear="none">This is what the 3 nodes say. One brick on each<br clear="none"></div><pre style="margin-top:4px;margin-bottom:4px;padding:8px;font-size:12px;line-height:1.50001;white-space:pre-wrap;border-radius:4px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,monospace">Node1getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.volume-client-2=0x000000000000000000000001trusted.afr.dirty=0x000000000000000000000000Node2getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.volume-client-2=0x000000000000000000000001trusted.afr.dirty=0x000000000000000000000000Node3(Arbiter)getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.dirty=0x000000000000000000000001</pre><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)">Since Node3(the arbiter) sees it as dirty and it looks like Node 1 and Node 2 have good copies, I was thinking of running the following on Node1 which I believe would tell Node 2 and Node 3 to sync from Node 1<br clear="none">I'd then kick off a heal on the volume<br clear="none"></div><pre style="margin-top:4px;margin-bottom:4px;padding:8px;font-size:12px;line-height:1.50001;white-space:pre-wrap;border-radius:4px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,monospace">setfattr -n trusted.afr.volume-client-1 -v 0x000000010000000000000000 /path/to/dirsetfattr -n trusted.afr.volume-client-2 -v 0x000000010000000000000000 /path/to/dir</pre><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)">client-0 is node 1, client-1 is node2 and client-2 is node 3. I've verified the hard links with gfid are in the xattrop directory<br clear="none">Is this the correct way to heal and resolve the issue?<span dir="ltr" style="font-size:13px;white-space:nowrap"> </span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)"><span dir="ltr" style="font-size:13px;white-space:nowrap"><br clear="none"></span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)"><span dir="ltr" style="font-size:13px;white-space:nowrap">Thanks</span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato,Slack-Fractions,appleLogo,sans-serif;font-size:15px;background-color:rgb(248,248,248)"><span dir="ltr" style="font-size:13px;white-space:nowrap">David</span></div></div></div>________<br clear="none"><br clear="none"><br clear="none"><br clear="none">Community Meeting Calendar:<br clear="none"><br clear="none">Schedule -<br clear="none">Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br clear="none">Bridge: <a rel="nofollow noopener noreferrer" shape="rect" href="https://meet.google.com/cpu-eiue-hvk" target="_blank">https://meet.google.com/cpu-eiue-hvk</a><br clear="none">Gluster-users mailing list<br clear="none"><a rel="nofollow noopener noreferrer" shape="rect" href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br clear="none"><a rel="nofollow noopener noreferrer" shape="rect" href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br clear="none"> </div> </blockquote></div></blockquote></div></div></div></div></div> </div> </blockquote></div></blockquote></div></div></div></div></div> </div> </div></blockquote></div></div>
</blockquote></div></div>
</div></div> </div> </blockquote></div></div></blockquote></div>