<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Result for a single ls on a dir with 10k directories inside (16*3
volume)</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-33e4f081-7fff-9afd-5c71-bdcbae1cffcc"><br>
<div dir="ltr" style="margin-left:0pt;" align="left">
<table
style="border:none;border-collapse:collapse;width:468pt;table-layout:fixed">
<colgroup><col width="199"><col width="113"><col><col></colgroup><tbody>
<tr style="height:21.75pt">
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Configuration</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Plain volume</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Parallel-readdir</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Proposed Solution</span></p>
</td>
</tr>
<tr style="height:0pt">
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Single Dir ls (Seconds)</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">-----</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">135</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">32.744</span></p>
</td>
</tr>
</tbody>
</table>
</div>
</b></p>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">It is showing 321% improvements.</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">Regards</div>
<div class="moz-cite-prefix">Rafi KC<br>
</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">On 27/05/20 11:22 am, RAFI KC wrote:<br>
</div>
<blockquote type="cite"
cite="mid:a4ddb402-65c6-a32b-eda4-821ec0b8ad26@redhat.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<p>Hi All,</p>
<p>I have been working on POC to improve readdirp performance
improvement. At the end of the experiment, The results are
showing promising result in performance, overall there is a 104%
improvement for full filesystem crawl compared to the existing
solution. Here is the short test numbers. The tests were carried
out in 16*3 setup with 1.5 Million dentries (Both files and
dir). The system also contains some empty directories. <b>In
the result the proposed solution is 287% faster than the plane
volume and 104% faster than the parallel-readdir based
solution.</b><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-8644ede0-7fff-09be-a0d9-5ee943340f2c"><br>
</b></p>
<div dir="ltr" style="margin-left:0pt;" align="left"><b
style="font-weight:normal;"
id="docs-internal-guid-8644ede0-7fff-09be-a0d9-5ee943340f2c">
<table
style="border:none;border-collapse:collapse;width:468pt;table-layout:fixed">
<colgroup><col width="199"><col width="113"><col><col></colgroup><tbody>
<tr style="height:0pt">
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Configuration</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Plain volume</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Parallel-readdir</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Proposed Solution</span></p>
</td>
</tr>
<tr style="height:0pt">
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">FS Crawl Time in Seconds</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">16497.523</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">8717.872</span></p>
</td>
<td style="border-left:solid #000000
1pt;border-right:solid #000000 1pt;border-bottom:solid
#000000 1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">4261.401 </span></p>
</td>
</tr>
</tbody>
</table>
</b></div>
<b style="font-weight:normal;"
id="docs-internal-guid-8644ede0-7fff-09be-a0d9-5ee943340f2c"> </b>
<p>In short, the basic idea behind the proposal is the efficient
managing of readdir buffer in gluster along with prefetching the
dentries for intelligent switch-over to the next buffer. The
detailed problem description, deign description and results are
available in the doc.<a
href="https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit"
moz-do-not-send="true">https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit</a>
<br>
</p>
<p><br>
</p>
<p><a class="moz-txt-link-freetext"
href="https://review.gluster.org/24469" moz-do-not-send="true">https://review.gluster.org/24469</a></p>
<p><a class="moz-txt-link-freetext"
href="https://review.gluster.org/24470" moz-do-not-send="true">https://review.gluster.org/24470</a><br>
</p>
<p><br>
</p>
<p>Regards</p>
<p>Rafi KC<br>
</p>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: <a class="moz-txt-link-freetext" href="https://bluejeans.com/441850968">https://bluejeans.com/441850968</a>
Gluster-devel mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Gluster-devel@gluster.org">Gluster-devel@gluster.org</a>
<a class="moz-txt-link-freetext" href="https://lists.gluster.org/mailman/listinfo/gluster-devel">https://lists.gluster.org/mailman/listinfo/gluster-devel</a>
</pre>
</blockquote>
</body>
</html>