Move away the file located onthe arbiter brick as it has different gfid and touch it(only if the software that consumes it is NOT sensitive to atime modification).<div><div><br></div><div>Best Regards,</div><div>Strahil Nikolov </div><div> <br> <blockquote style="margin: 0 0 20px 0;"> <div style="font-family:Roboto, sans-serif; color:#6D00F6;"> <div>On Wed, Feb 22, 2023 at 13:09, David Dolan</div><div><daithidolan@gmail.com> wrote:</div> </div> <div style="padding: 10px 0 0 20px; margin: 10px 0 0 0; border-left: 1px solid #6D00F6;"> <div id="yiv9993888487"><div><div dir="ltr">Hi Strahil,<div><br clear="none"></div><div>The output in my previous email showed the directory the file is located in with a different GFID on the Arbiter node compared with the bricks on the other nodes.</div><div><br clear="none"></div><div>Based on that, do you know what my next step should be?</div><div><br clear="none"></div><div>Thanks</div><div>David</div><div><br clear="none"></div></div><br clear="none"><div id="yiv9993888487yqt02617" class="yiv9993888487yqt4574699598"><div class="yiv9993888487gmail_quote"><div dir="ltr" class="yiv9993888487gmail_attr">On Wed, 15 Feb 2023 at 09:21, David Dolan <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:daithidolan@gmail.com" target="_blank" href="mailto:daithidolan@gmail.com">daithidolan@gmail.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;" class="yiv9993888487gmail_quote"><div dir="ltr"><div dir="ltr">sorry I didn't receive the previous email.<div>I've run the command on all 3 nodes(bricks). See below. The directory only has one file.</div><div>On the Arbiter, the file doesn't exist and the directory the file should be in has a different GFID than the bricks on the other nodes</div><div><br clear="none"></div><div>Node 1 Brick</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2/file</div><div>trusted.gfid=0x7b1aa40dd1e64b7b8aac7fc6bcbc9e9b<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2<br clear="none"></div><div>trusted.gfid=0xdc99ac0db85d4b1c8a6af57a71bbe22c<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Node 2 Brick</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2/file<br clear="none"></div><div>trusted.gfid=0x7b1aa40dd1e64b7b8aac7fc6bcbc9e9b<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2</div><div>trusted.gfid=0xdc99ac0db85d4b1c8a6af57a71bbe22c<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Node 3 Brick (Arbiter)</div><div>Path to file doesn't exist</div><div>getfattr -d -m . -e hex /path_on_brick/subdir1/subdir2<br clear="none"></div><div>trusted.gfid=0x51cca97ac2974ceb9322fe21e6f8ea91<br clear="none"></div><div>getfattr -d -m . -e hex /path_on_brick/subdir1<br clear="none"></div><div>trusted.gfid=0x2aa1fe9e65094e6188fc91a6d16dd2c4<br clear="none"></div><div><br clear="none"></div><div>Thanks</div><div>David</div></div><br clear="none"><div class="yiv9993888487gmail_quote"><div dir="ltr" class="yiv9993888487gmail_attr">On Tue, 14 Feb 2023 at 20:38, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;" class="yiv9993888487gmail_quote"><div> I guess you didn't receive my last e-mail.<br clear="none">Use getfattr and identify if the gfid mismatch. If yes, move away the mismatched one.<br clear="none">In order a dir to heal, you have to fix all files inside it before it can be healed.<br clear="none"><br clear="none">Best Regards,<br clear="none">Strahil Nikolov </div> <div style="margin:10px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"> <div style="font-family:Helvetica, Arial, sans-serif;font-size:13px;color:rgb(38,40,42);"> <div> В вторник, 14 февруари 2023 г., 14:04:31 ч. Гринуич+2, David Dolan <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:daithidolan@gmail.com" target="_blank" href="mailto:daithidolan@gmail.com">daithidolan@gmail.com</a>> написа: </div> <div><br clear="none"></div> <div><br clear="none"></div> <div><div id="yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711"><div><div dir="ltr">I've touched the directory one level above the directory with the I\O issue as the one above that is the one showing as dirty.<div>It hasn't healed. Should the self heal daemon automatically kick in here?</div><div><br clear="none"></div><div>Is there anything else I can do?</div><div><br clear="none"></div><div>Thanks</div><div>David</div></div><br clear="none"><div id="yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711yqt33930"><div><div dir="ltr">On Tue, 14 Feb 2023 at 07:03, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">You can always mount it locally on any of the gluster nodes.<div><br clear="none"></div><div>Best Regards,</div><div>Strahil Nikolov <br clear="none"> <br clear="none"> <blockquote style="margin:0px 0px 20px;"> <div style="font-family:Roboto, sans-serif;color:rgb(109,0,246);"> <div>On Mon, Feb 13, 2023 at 18:13, David Dolan</div><div><<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:daithidolan@gmail.com" target="_blank" href="mailto:daithidolan@gmail.com">daithidolan@gmail.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246);"> <div id="yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743"><div><div dir="ltr"><div dir="ltr">HI Strahil,<div><br clear="none"></div><div>Thanks for that. It's the first time I've been in this position, so I'm learning as I go along.</div><div><br clear="none"></div><div>Unfortunately I can't go into the directory on the client side as I get an input/output error</div><div>Input/output error<br clear="none"></div><div>d????????? ? ? ? ? ? 01<br clear="none"></div><div><br clear="none"></div><div>Thanks</div><div>David</div><div><br clear="none"></div></div><br clear="none"><div id="yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743yqt14550"><div><div dir="ltr">On Sun, 12 Feb 2023 at 20:29, Strahil Nikolov <<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:hunter86_bg@yahoo.com" target="_blank" href="mailto:hunter86_bg@yahoo.com">hunter86_bg@yahoo.com</a>> wrote:<br clear="none"></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">Setting blame on client-1 and client-2 will make a bigger mess.<div>Can't you touch the affected file from the FUSE mount point ?</div><div><br clear="none"></div><div>Best Regards,</div><div>Strahil Nikolov <br clear="none"> <br clear="none"> <blockquote style="margin:0px 0px 20px;"> <div style="font-family:Roboto, sans-serif;color:rgb(109,0,246);"> <div>On Tue, Feb 7, 2023 at 14:42, David Dolan</div><div><<a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:daithidolan@gmail.com" target="_blank" href="mailto:daithidolan@gmail.com">daithidolan@gmail.com</a>> wrote:</div> </div> <div style="padding:10px 0px 0px 20px;margin:10px 0px 0px;border-left:1px solid rgb(109,0,246);"> <div id="yiv9993888487m_-4493405580498555040m_-8138453635600812094yiv7346299711m_404956839328620909yiv6590544743m_4113625973783826226yiv8260473842"><div dir="ltr"><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);">Hi All. </div><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);"><br clear="none"></div><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);">Hoping you can help me with a healing problem. I have one file which didn't self heal.<br clear="none">it looks to be a problem with a directory in the path as one node says it's dirty. I have a replica volume with arbiter<br clear="none">This is what the 3 nodes say. One brick on each<br clear="none"></div><pre style="margin-top:4px;margin-bottom:4px;padding:8px;font-size:12px;line-height:1.50001;white-space:pre-wrap;border-radius:4px;color:rgb(29,28,29);font-family:Monaco, Menlo, Consolas, monospace;">Node1getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.volume-client-2=0x000000000000000000000001trusted.afr.dirty=0x000000000000000000000000Node2getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.volume-client-2=0x000000000000000000000001trusted.afr.dirty=0x000000000000000000000000Node3(Arbiter)getfattr -d -m . -e hex /path/to/dir | grep afrgetfattr: Removing leading '/' from absolute path namestrusted.afr.dirty=0x000000000000000000000001</pre><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);">Since Node3(the arbiter) sees it as dirty and it looks like Node 1 and Node 2 have good copies, I was thinking of running the following on Node1 which I believe would tell Node 2 and Node 3 to sync from Node 1<br clear="none">I'd then kick off a heal on the volume<br clear="none"></div><pre style="margin-top:4px;margin-bottom:4px;padding:8px;font-size:12px;line-height:1.50001;white-space:pre-wrap;border-radius:4px;color:rgb(29,28,29);font-family:Monaco, Menlo, Consolas, monospace;">setfattr -n trusted.afr.volume-client-1 -v 0x000000010000000000000000 /path/to/dirsetfattr -n trusted.afr.volume-client-2 -v 0x000000010000000000000000 /path/to/dir</pre><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);">client-0 is node 1, client-1 is node2 and client-2 is node 3. I've verified the hard links with gfid are in the xattrop directory<br clear="none">Is this the correct way to heal and resolve the issue?<span dir="ltr" style="font-size:13px;white-space:nowrap;"> </span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);"><span dir="ltr" style="font-size:13px;white-space:nowrap;"><br clear="none"></span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);"><span dir="ltr" style="font-size:13px;white-space:nowrap;">Thanks</span></div><div style="color:rgb(29,28,29);font-family:Slack-Lato, Slack-Fractions, appleLogo, sans-serif;font-size:15px;background-color:rgb(248,248,248);"><span dir="ltr" style="font-size:13px;white-space:nowrap;">David</span></div></div></div>________<br clear="none"><br clear="none"><br clear="none"><br clear="none">Community Meeting Calendar:<br clear="none"><br clear="none">Schedule -<br clear="none">Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br clear="none">Bridge: <a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://meet.google.com/cpu-eiue-hvk">https://meet.google.com/cpu-eiue-hvk</a><br clear="none">Gluster-users mailing list<br clear="none"><a rel="nofollow noopener noreferrer" shape="rect" ymailto="mailto:Gluster-users@gluster.org" target="_blank" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br clear="none"><a rel="nofollow noopener noreferrer" shape="rect" target="_blank" href="https://lists.gluster.org/mailman/listinfo/gluster-users">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br clear="none"> </div> </blockquote></div></blockquote></div></div></div></div></div> </div> </blockquote></div></blockquote></div></div></div></div></div> </div> </div></blockquote></div></div>
</blockquote></div></div>
</div></div> </div> </blockquote></div></div>