<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p><br>
</p>
<div class="moz-cite-prefix">On 11/4/19 2:41 PM, Riccardo Murri
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJGE3zXPzBKpJF=_ZEsuW1G6ss3TUzKHbpdQdkueB7AKMOZbYg@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">Hello Amar,
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Can you please check the profile info [1] ? That may give some hints.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
I am attaching the output of `sudo gluster volume profile info` as a text file
to preserve formatting. This covers the time from Friday night to
Monday morning;
during this time the cluster has been the target of a an `rsync` command copying
a large directory tree (14TB; it's taking more than 2 weeks now...).
My take from cursorily reading the profile:
- metadata operations (opendir, create, rename) seem to have very high
latency (up to 25 seconds for a rename!)</pre>
</blockquote>
<p><br>
</p>
<p>We have identified a performance drop with ctime and rename
operations. This is because ctime stores some metadata information
as an extended attributes which sometimes exceeds the default
inode size. In such scenarios the additional xattrs won't fit into
the default size. This will result in additional blocks to be used
which will effect the latency.</p>
<p>Is it possible for you to repeat the test by disabling ctime or
increasing the inode size to a higher value say 1024? <br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CAJGE3zXPzBKpJF=_ZEsuW1G6ss3TUzKHbpdQdkueB7AKMOZbYg@mail.gmail.com">
<pre class="moz-quote-pre" wrap="">
- large block size (>128kB) are rare; most blocks seems to be 8kB,
16kB, or 128kB.
I personally do not see any obvious culprit or optimization
opportunities; is there something I'm missing out that catches the eye
of someone more experienced?
Thanks,
R
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">________
Community Meeting Calendar:
APAC Schedule -
Every 2nd and 4th Tuesday at 11:30 AM IST
Bridge: <a class="moz-txt-link-freetext" href="https://bluejeans.com/118564314">https://bluejeans.com/118564314</a>
NA/EMEA Schedule -
Every 1st and 3rd Tuesday at 01:00 PM EDT
Bridge: <a class="moz-txt-link-freetext" href="https://bluejeans.com/118564314">https://bluejeans.com/118564314</a>
Gluster-users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-users@gluster.org">Gluster-users@gluster.org</a>
<a class="moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-users">https://lists.gluster.org/mailman/listinfo/gluster-users</a>
</pre>
</blockquote>
</body>
</html>