<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 03/10/2017 10:32 PM, Luca Gervasi
wrote:<br>
</div>
<blockquote
cite="mid:CAPJi5KOqiUuHN6yeZ5aV-ms+NMp10ppZdYMQc+T8+PiSFRrGMw@mail.gmail.com"
type="cite">
<div dir="ltr">Hi,
<div>I'm Andrea's collegue. I'd like to add that we have no
trusted.afr xattr on the root folder</div>
</div>
</blockquote>
Just to confirm, this would be 'includes2013' right?<br>
<blockquote
cite="mid:CAPJi5KOqiUuHN6yeZ5aV-ms+NMp10ppZdYMQc+T8+PiSFRrGMw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div> where those files are located and every file seems to be
clean on each brick.</div>
<div>You can find another example file's xattr here: <a
moz-do-not-send="true"
href="https://nopaste.me/view/3c2014ac">https://nopaste.me/view/3c2014ac</a></div>
<div>Here a listing: <a moz-do-not-send="true"
href="https://nopaste.me/view/eb4430a2">https://nopaste.me/view/eb4430a2</a></div>
<div>This behavior causes the directory which contains those
files undeletable (we had to clear them up on brick level,
clearing all the hardlinks too).</div>
<div>This issue is visible on fuse mounted volumes while it's
not noticeable when mounted in NFS through ganesha.</div>
<div><br>
</div>
</div>
</blockquote>
Could you provide the complete output of `gluster volume info`? I
want to find out which bricks constitute a replica pair.<br>
<br>
Also could you change the diagnostics.client-log-level to DEBUG
temporarily, do an `ls <directory where you see duplicate
entries>` on the fuse mount and share the corresponding mount
log?<br>
Thanks,<br>
Ravi<br>
<br>
<blockquote
cite="mid:CAPJi5KOqiUuHN6yeZ5aV-ms+NMp10ppZdYMQc+T8+PiSFRrGMw@mail.gmail.com"
type="cite">
<div dir="ltr">
<div>Thanks a lot.</div>
<div><br>
</div>
<div>Luca Gervasi</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Fri, 10 Mar 2017 at 17:41 Andrea Fogazzi <<a
moz-do-not-send="true" href="mailto:foggy@fogazzi.com">foggy@fogazzi.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr" class="gmail_msg">
<div id="m_-7359613590814742918divtagdefaultwrapper"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
dir="ltr" class="gmail_msg">
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<p class="gmail_msg">Hi community,</p>
<p class="gmail_msg">we ran an extensive issue on
our installation of gluster 3.10, which we did
upgraded from 3.8.8 (it's a distribute+replicate, 5
nodes, 3 bricks in replica 2+1 quorum); recently we
noticed a frequent issue where files get duplicated on
the some of the directories; this is visible on the
fuse mount points (RW), but not on the NFS/Ganesha
(RO) mount points.</p>
<p class="gmail_msg"><br class="gmail_msg">
</p>
<p class="gmail_msg">A sample of an ll output:</p>
<p class="gmail_msg"><br class="gmail_msg">
</p>
<div class="gmail_msg">---------T 1 48 web_rw 0 Mar
10 11:57 paginazione.shtml</div>
<div class="gmail_msg">-rw-rw-r-- 1 48 web_rw 272 Feb
18 22:00 paginazione.shtml</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<br class="gmail_msg">
</div>
As you can see, the file is listed twice, but only one
of the two is good (the name is identical, we verified
that no spurious/hidden characters are present in the
name); the issue maybe is related on how we uploaded the
files on the file system, via incremental rsync on the
fuse mount.</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<br class="gmail_msg">
</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
Do anyone have suggestion on how it can happen, how to
solve existing duplication or how to prevent to happen
anymore.</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<br class="gmail_msg">
</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
Thanks in advance.</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
Best regards,</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
andrea</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<br class="gmail_msg">
</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<div class="gmail_msg">Options Reconfigured:</div>
<div class="gmail_msg">performance.cache-invalidation:
true</div>
<div class="gmail_msg">cluster.favorite-child-policy:
mtime</div>
<div class="gmail_msg">features.cache-invalidation: 1</div>
<div class="gmail_msg">network.inode-lru-limit: 90000</div>
<div class="gmail_msg">performance.cache-size: 1024MB</div>
<div class="gmail_msg">storage.linux-aio: on</div>
<div class="gmail_msg">nfs.outstanding-rpc-limit: 64</div>
<div class="gmail_msg">storage.build-pgfid: on</div>
<div class="gmail_msg">cluster.server-quorum-type:
server</div>
<div class="gmail_msg">cluster.self-heal-daemon: enable</div>
<div class="gmail_msg">performance.nfs.io-cache: on</div>
<div class="gmail_msg">performance.client-io-threads: on</div>
<div class="gmail_msg">performance.nfs.stat-prefetch: on</div>
<div class="gmail_msg">performance.nfs.io-threads: on</div>
<div class="gmail_msg">diagnostics.latency-measurement:
on</div>
<div class="gmail_msg">diagnostics.count-fop-hits: on</div>
<div class="gmail_msg">performance.md-cache-timeout: 1</div>
<div class="gmail_msg">performance.io-thread-count: 16</div>
<div class="gmail_msg">performance.high-prio-threads: 32</div>
<div class="gmail_msg">performance.normal-prio-threads:
32</div>
<div class="gmail_msg">performance.low-prio-threads: 32</div>
<div class="gmail_msg">performance.least-prio-threads: 1</div>
<div class="gmail_msg">nfs.acl: off</div>
<div class="gmail_msg">nfs.rpc-auth-unix: off</div>
<div class="gmail_msg">diagnostics.client-log-level:
ERROR</div>
<div class="gmail_msg">diagnostics.brick-log-level:
ERROR</div>
<div class="gmail_msg">cluster.lookup-unhashed: auto</div>
<div class="gmail_msg">performance.nfs.quick-read: on</div>
<div class="gmail_msg">performance.nfs.read-ahead: on</div>
<div class="gmail_msg">cluster.quorum-type: auto</div>
<div class="gmail_msg">cluster.locking-scheme: granular</div>
<div class="gmail_msg">cluster.data-self-heal-algorithm:
full</div>
<div class="gmail_msg">transport.address-family: inet</div>
<div class="gmail_msg">performance.readdir-ahead: on</div>
<div class="gmail_msg">nfs.disable: on</div>
<div class="gmail_msg">cluster.lookup-optimize: on</div>
<div class="gmail_msg">cluster.readdir-optimize: on</div>
<div class="gmail_msg">performance.read-ahead: off</div>
<div class="gmail_msg">performance.write-behind-window-size:
1MB</div>
<div class="gmail_msg">client.event-threads: 4</div>
<div class="gmail_msg">server.event-threads: 16</div>
<div class="gmail_msg">cluster.granular-entry-heal:
enable</div>
<div class="gmail_msg">performance.parallel-readdir: on</div>
<div class="gmail_msg">cluster.server-quorum-ratio: 51</div>
</div>
</div>
</div>
<div dir="ltr" class="gmail_msg">
<div id="m_-7359613590814742918divtagdefaultwrapper"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
dir="ltr" class="gmail_msg">
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<br class="gmail_msg">
</div>
<div id="m_-7359613590814742918divtagdefaultwrapper"
dir="ltr"
style="font-size:12pt;color:#000000;font-family:Calibri,Arial,Helvetica,sans-serif"
class="gmail_msg">
<p class="gmail_msg"><br class="gmail_msg">
</p>
<p class="gmail_msg"><br class="gmail_msg">
</p>
<div id="m_-7359613590814742918Signature"
class="gmail_msg"><span
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"
class="gmail_msg">Andrea Fogazzi</span><br
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"
class="gmail_msg">
<br class="gmail_msg">
<div class="gmail_msg"><span
style="color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px"
class="gmail_msg"></span></div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="http://lists.gluster.org/mailman/listinfo/gluster-users">http://lists.gluster.org/mailman/listinfo/gluster-users</a></pre>
</blockquote>
<p><br>
</p>
</body>
</html>