<div>I did a find on this inode number and I could find the file but only on node1 (nothing on node2 and the new arbiternode). Here is an ls -lai of the file itself on node1:<br></div><div><br></div><div>-rw-r--r-- 1 www-data www-data 32 Jun 19 17:42 fileKey<br></div><div><br></div><div>As you can see it is a 32 bytes file and as you suggested I ran a "stat" on this very same file through a glusterfs mount (using fuse) but unfortunately nothing happened. The GFID is still being displayed to be healed. Just in case here is the output of the stat:<br></div><div><br></div><div> File: ‘fileKey’<br></div><div> Size: 32 Blocks: 1 IO Block: 131072 regular file<br></div><div>Device: 1eh/30d Inode: 12086351742306673840 Links: 1<br></div><div>Access: (0644/-rw-r--r--) Uid: ( 33/www-data) Gid: ( 33/www-data)<br></div><div>Access: 2017-06-19 17:42:35.339773495 +0200<br></div><div>Modify: 2017-06-19 17:42:35.343773437 +0200<br></div><div>Change: 2017-06-19 17:42:35.343773437 +0200<br></div><div>Birth: -<br></div><div><br></div><div>What else can I do or try in order to fix this situation?<br></div><div><br></div><div class="protonmail_signature_block protonmail_signature_block-empty"><div class="protonmail_signature_block-user protonmail_signature_block-empty"><div><br></div></div><div class="protonmail_signature_block-proton protonmail_signature_block-empty"><br></div></div><div><br></div><blockquote class="protonmail_quote" type="cite"><div>-------- Original Message --------<br></div><div>Subject: Re: [Gluster-users] Possible stale .glusterfs/indices/xattrop file?<br></div><div>Local Time: July 31, 2017 3:27 AM<br></div><div>UTC Time: July 31, 2017 1:27 AM<br></div><div>From: ravishankar@redhat.com<br></div><div>To: mabi <mabi@protonmail.ch><br></div><div>Gluster Users <gluster-users@gluster.org><br></div><div><br></div><div> <br></div><p><br></p><div><br></div><div class="moz-cite-prefix">On 07/30/2017 02:24 PM, mabi wrote:<br></div><blockquote type="cite"><div>Hi Ravi,<br></div><div><br></div><div>Thanks for your hints. Below you will find the answer to your
questions.<br></div><div><br></div><div>First I tried to start the healing process by running:<br></div><div><br></div><div>gluster volume heal myvolume<br></div><div><br></div><div>and then as you suggested watch the output of the
glustershd.log file but nothing appeared in that log file after
running the above command. I checked the files which need to be
healing using the "heal <volume> info" command and it
still shows that very same GFID on node2 to be healed. So
nothing changed here.<br></div><div><br></div><div>The file
/data/myvolume/brick/.glusterfs/indices/xattrop/29e0d13e-1217-41cc-9bda-1fbbf781c397
is only on node2 and not on my nod1 nor on my arbiternode. This
file seems to be a regular file and not a symlink. Here is the
output of the stat command on it from my node2:<br></div><div><br></div><div> File:
‘/data/myvolume/brick/.glusterfs/indices/xattrop/29e0d13e-1217-41cc-9bda-1fbbf781c397’<br></div><div> Size: 0 Blocks: 1 IO Block: 512 regular
empty file<br></div><div>Device: 25h/37d Inode: 2798404 Links: 2<br></div></blockquote><div><br></div><div>Okay, link count of 2 means there is a hardlink somewhere on the
brick. Try the find command again. I see that the inode number is
2798404, not the one you shared in your first mail. Once you find
the path to the file, do a stat of the file from mount. This should
create the entry in the other 2 bricks and do the heal. But FWIW,
this seems to be a zero byte file.<br></div><div> <br></div><div> Regards,<br></div><div> Ravi<br></div><div> <br></div><blockquote type="cite"><div>Access: (0000/----------) Uid: ( 0/ root) Gid: (
0/ root)<br></div><div>Access: 2017-04-28 22:51:15.215775269 +0200<br></div><div>Modify: 2017-04-28 22:51:15.215775269 +0200<br></div><div>Change: 2017-07-30 08:39:03.700872312 +0200<br></div><div>Birth: -<br></div><div><br></div><div>I hope this is enough info for a starter, else let me know if
you need any more info. I would be glad to resolve this weird
file which needs to be healed but can not.<br></div><div><br></div><div>Best regards,<br></div><div>Mabi<br></div><div class="protonmail_signature_block
protonmail_signature_block-empty"><div class="protonmail_signature_block-user
protonmail_signature_block-empty"><div><br></div></div><div class="protonmail_signature_block-proton
protonmail_signature_block-empty"><br></div></div><div><br></div><blockquote type="cite" class="protonmail_quote"><div>-------- Original Message --------<br></div><div>Subject: Re: [Gluster-users] Possible stale
.glusterfs/indices/xattrop file?<br></div><div>Local Time: July 30, 2017 3:31 AM<br></div><div>UTC Time: July 30, 2017 1:31 AM<br></div><div>From: <a href="mailto:ravishankar@redhat.com" class="moz-txt-link-abbreviated" rel="noreferrer nofollow noopener">ravishankar@redhat.com</a><br></div><div>To: mabi <a href="mailto:mabi@protonmail.ch" class="moz-txt-link-rfc2396E" rel="noreferrer nofollow noopener"><mabi@protonmail.ch></a>, Gluster Users <a href="mailto:gluster-users@gluster.org" class="moz-txt-link-rfc2396E" rel="noreferrer nofollow noopener"><gluster-users@gluster.org></a><br></div><div><br></div><div><br></div><p><br></p><div><br></div><div class="moz-cite-prefix">On 07/29/2017 04:36 PM, mabi wrote:<br></div><blockquote type="cite"><div>Hi,<br></div><div><br></div><div>Sorry for mailing again but as mentioned in my previous
mail, I have added an arbiter node to my replica 2 volume
and it seem to have gone fine except for the fact that there
is one single file which needs healing and does not get
healed as you can see here from the output of a "heal info":<br></div><div><br></div><div>Brick node1.domain.tld:/data/myvolume/brick<br></div><div>Status: Connected<br></div><div>Number of entries: 0<br></div><div><br></div><div>Brick node2.domain.tld:/data/myvolume/brick<br></div><div><gfid:29e0d13e-1217-41cc-9bda-1fbbf781c397><br></div><div>Status: Connected<br></div><div>Number of entries: 1<br></div><div><br></div><div>Brick
arbiternode.domain.tld:/srv/glusterfs/myvolume/brick<br></div><div>Status: Connected<br></div><div>Number of entries: 0<br></div><div><br></div><div>On my node2 the respective .glusterfs/indices/xattrop
directory contains two files as you can see below:<br></div><div><br></div><div>ls -lai /data/myvolume/brick/.glusterfs/indices/xattrop<br></div><div>total 76180<br></div><div> 10 drw------- 2 root root 4 Jul 29 12:15 .<br></div><div> 9 drw------- 5 root root 5 Apr 28 22:15 ..<br></div><div>2798404 ---------- 2 root root 0 Apr 28 22:51
29e0d13e-1217-41cc-9bda-1fbbf781c397<br></div><div>2798404 ---------- 2 root root 0 Apr 28 22:51
xattrop-6fa49ad5-71dd-4ec2-9246-7b302ab92d38<br></div><div class="protonmail_signature_block
protonmail_signature_block-empty"><div class="protonmail_signature_block-user
protonmail_signature_block-empty"><div><br></div></div><div class="protonmail_signature_block-proton
protonmail_signature_block-empty"><br></div></div><div><br></div><div>I tried to find the real file on my brick where this
xattrop file points to using its inode number (command: find
/data/myvolume/brick/data -inum 8394642) but it does not
find any associated file.<br></div><div><br></div><div>So my question here is, is it possible that this is a
stale file which just forgot to get deleted from the
indices/xattrop file by gluster for some unknown reason? If
yes is it safe for me to delete these two files? or what
would be the correct process in that case?<br></div></blockquote><div>The 'xattrop-6fa...' is the base entry. gfids of files that
need heal are hard linked to this entry, so nothing needs to
be done for it. But you need to find out why '29e0d13...' is
not healing. Launch the heal and observe the glustershd logs
for errors. I suppose the inode number for
.glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 is what
is 8394642. Is
.glusterfs/29/e0/29e0d13e-1217-41cc-9bda-1fbbf781c397 a
regular file or symlink? Does it exist in the other 2 bricks?
What is the link count (as seen from stat <file>)?<br></div><div>-Ravi<br></div><div><br></div><blockquote type="cite"><div><br></div><div>Thank you for your input.<br></div><div>Mabi<br></div><div><br></div><div><br></div><pre wrap="">_______________________________________________
Gluster-users mailing list
<a rel="noreferrer nofollow noopener" class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a rel="noreferrer nofollow noopener" class="moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a>
<br></pre></blockquote></blockquote><div><br></div></blockquote></blockquote><div><br></div>