<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi All,</p>
<p>I have been working on POC to improve readdirp performance
improvement. At the end of the experiment, The results are showing
promising result in performance, overall there is a 104%
improvement for full filesystem crawl compared to the existing
solution. Here is the short test numbers. The tests were carried
out in 16*3 setup with 1.5 Million dentries (Both files and dir).
The system also contains some empty directories. <b>In the result
the proposed solution is 287% faster than the plane volume and
104% faster than the parallel-readdir based solution.</b><br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-8644ede0-7fff-09be-a0d9-5ee943340f2c"><br>
</b></p>
<div dir="ltr" style="margin-left:0pt;" align="left">
<table
style="border:none;border-collapse:collapse;width:468pt;table-layout:fixed">
<colgroup><col width="199"><col width="113"><col><col></colgroup><tbody>
<tr style="height:0pt">
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Configuration</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Plain volume</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Parallel-readdir</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">Proposed Solution</span></p>
</td>
</tr>
<tr style="height:0pt">
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">FS Crawl Time in Seconds</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">16497.523</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">8717.872</span></p>
</td>
<td style="border-left:solid #000000 1pt;border-right:solid
#000000 1pt;border-bottom:solid #000000
1pt;border-top:solid #000000
1pt;vertical-align:top;padding:5pt 5pt 5pt
5pt;overflow:hidden;overflow-wrap:break-word;">
<p dir="ltr" style="line-height:1.2;text-align:
center;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:10pt;font-family:Arial;color:#000000;background-color:#ffffff;font-weight:700;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre;white-space:pre-wrap;">4261.401 </span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p>In short, the basic idea behind the proposal is the efficient
managing of readdir buffer in gluster along with prefetching the
dentries for intelligent switch-over to the next buffer. The
detailed problem description, deign description and results are
available in the doc.<a
href="https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit">https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit</a>
<br>
</p>
<p><br>
</p>
<p>If anybody can help with the testing on a different kind of
workloads, I would be very happy to assist. If wanted to test the
patch and run a performance test on your setup, I could help with
back-porting the patch to the version of your choice.<br>
</p>
<p><br>
</p>
<p><a class="moz-txt-link-freetext" href="https://review.gluster.org/24469">https://review.gluster.org/24469</a></p>
<p><a class="moz-txt-link-freetext" href="https://review.gluster.org/24470">https://review.gluster.org/24470</a><br>
</p>
<p><br>
</p>
<p>Regards</p>
<p>Rafi KC<br>
</p>
</body>
</html>