<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On 9 July 2018 at 14:20, Niels de Vos <span dir="ltr"><<a href="mailto:ndevos@redhat.com" target="_blank">ndevos@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Mon, Jul 09, 2018 at 01:30:23PM +0530, Nithya Balachandran wrote:<br>
> Hi Hans,<br>
> <br>
> Another user has reported something similar and we are still debugging this.<br>
> <br>
> Would you mind taking a tcpdump of the client while listing the directory<br>
> from a FUSE client and sending it to me? Please use<br>
> tcpdump -i any -s 0 -w /var/tmp/dirls.pcap tcp and not port 22<br>
<br>
</span>Last week I came across a patch that fixes a bug in the FUSE kernel<br>
module:<br>
<a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c6cdd51404b7ac12dd95173ddfc548c59ecf037f" rel="noreferrer" target="_blank">https://git.kernel.org/pub/<wbr>scm/linux/kernel/git/torvalds/<wbr>linux.git/commit/?id=<wbr>c6cdd51404b7ac12dd95173ddfc548<wbr>c59ecf037f</a><br>
<br>
This was included in linux-4.14 but has been marked for backporting to<br>
older kernels as well. If the description matches the experienced<br>
behaviour, running with an updates (or patched) kernel may help.<br>
<span class="HOEnZb"><font color="#888888"><br></font></span></blockquote><div><br></div><div>This is interesting but it won't help in this case. The issue is with Gluster.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="HOEnZb"><font color="#888888">
Niels<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
> <br>
> <br>
> Also, please send the output of gluster volume info and gluster volume get<br>
> <volname> all.<br>
> <br>
> Thanks,<br>
> Nithya<br>
> <br>
> <br>
> On 9 July 2018 at 12:51, Hans Henrik Happe <<a href="mailto:happe@nbi.dk">happe@nbi.dk</a>> wrote:<br>
> <br>
> > Hi,<br>
> ><br>
> > After an upgrade from 3.7 -> 3.10 -> 3.12.9 that seemed to go smoothly,<br>
> > we have experienced missing files and dirs when listing directories.<br>
> ><br>
> > We are using a distributed setup with 20 bricks (no redundance from<br>
> > glusterfs).<br>
> ><br>
> > The dirs and files can be referenced directly, but does not show up in<br>
> > listings (readdir, i.e. ls). Renaming them works, but they still does<br>
> > not show up.<br>
> ><br>
> > The first time we discovered this, we noticed that files slowly<br>
> > reappeared and finally all were there. After that we started a<br>
> > fix-layout which is still running (5mio dirs). After this we would<br>
> > compare brick files to the mounted fs.<br>
> ><br>
> > Yesterday we again discovered some missing files in a dir. After some<br>
> > poking around we found that all missing files were located on the same<br>
> > brick.<br>
> ><br>
> > Comparing dir xattr did not give us a clue:<br>
> ><br>
> ><br>
> > Brick with missing files:<br>
> ><br>
> > # getfattr -m . -d -e hex backup<br>
> > # file: backup<br>
> > trusted.gfid=<wbr>0x8613f6e0317141918b42d8c8063f<wbr>fbce<br>
> > trusted.glusterfs.dht=<wbr>0x0000000100000000b2169fa3bf11<wbr>b4e7<br>
> > trusted.glusterfs.quota.<wbr>6e0ab807-6eed-4af1-92b8-<wbr>0db3ca7a19e0.contri.1=<br>
> > 0x00000000d65e3400000000000000<wbr>00750000000000000001<br>
> > trusted.glusterfs.quota.dirty=<wbr>0x3000<br>
> > trusted.glusterfs.quota.size.<wbr>1=<wbr>0x00000000d65e3400000000000000<br>
> > 00750000000000000001<br>
> ><br>
> > Other brick:<br>
> ><br>
> > # getfattr -m . -d -e hex backup<br>
> > # file: backup<br>
> > trusted.gfid=<wbr>0x8613f6e0317141918b42d8c8063f<wbr>fbce<br>
> > trusted.glusterfs.dht=<wbr>0x0000000100000000bf11b4e8cbe5<wbr>e488<br>
> > trusted.glusterfs.quota.<wbr>6e0ab807-6eed-4af1-92b8-<wbr>0db3ca7a19e0.contri.1=<br>
> > 0x00000000b03aa800000000000000<wbr>00700000000000000001<br>
> > trusted.glusterfs.quota.dirty=<wbr>0x3000<br>
> > trusted.glusterfs.quota.size.<wbr>1=<wbr>0x00000000b03aa800000000000000<br>
> > 00700000000000000001<br>
> ><br>
> ><br>
> > Anyone who experienced this or have some clues to what might be wrong?<br>
> ><br>
> > Cheers,<br>
> > Hans Henrik<br>
> > ______________________________<wbr>_________________<br>
> > Gluster-users mailing list<br>
> > <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> > <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
> ><br>
<br>
> ______________________________<wbr>_________________<br>
> Gluster-users mailing list<br>
> <a href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a><br>
> <a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/<wbr>mailman/listinfo/gluster-users</a><br>
<br>
</div></div></blockquote></div><br></div></div>