<div dir="ltr"><div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 27, 2020 at 10:35 AM RAFI KC <<a href="mailto:rkavunga@redhat.com">rkavunga@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Hi Felix,</p>
<p>Thanks for your mail. I will test it more to make sure that it
doesn't break anything. Also I have added a configuration key for
easier switching to the older code in case if there is any
problem. If you can help me in any manner in testing or
performance numbers, please let me know.</p></div></blockquote><div><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">A scratch build allowing the community to test this might be beneficial.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Y.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>
<p><br>
</p>
<p>Regards</p>
<p>Rafi KC<br>
</p>
<div>On 27/05/20 12:37 pm, Felix Kölzow
wrote:<br>
</div>
<blockquote type="cite">
<p>Dear Rafi,</p>
<p><br>
</p>
<p>thanks for your effort. I think this is of great interest of
many gluster users. Thus, I would really encourage you to <br>
</p>
<p>test and to further improve this feature. Maybe it is
beneficial to create a certain guideline which things should be
tested</p>
<p>to make this feature really ready for productive use.</p>
<p><br>
</p>
<p>Thanks in advance.</p>
<p>Felix<br>
</p>
<div>On 27/05/2020 07:56, RAFI KC wrote:<br>
</div>
<blockquote type="cite">
<p>Hi All,</p>
<p>I have been working on POC to improve readdirp performance
improvement. At the end of the experiment, The results are
showing promising result in performance, overall there is a
104% improvement for full filesystem crawl compared to the
existing solution. Here is the short test numbers. The tests
were carried out in 16*3 setup with 1.5 Million dentries (Both
files and dir). The system also contains some empty
directories. <b>In the result the proposed solution is 287%
faster than the plane volume and 104% faster than the
parallel-readdir based solution.</b><br>
</p>
<p><b id="gmail-m_2194463795343463852docs-internal-guid-8644ede0-7fff-09be-a0d9-5ee943340f2c"><br>
</b></p>
<div dir="ltr">
<table>
<colgroup><col width="199"><col width="113"><col><col></colgroup><tbody>
<tr>
<td>
<p dir="ltr"><span>Configuration</span></p>
</td>
<td>
<p dir="ltr"><span>Plain volume</span></p>
</td>
<td>
<p dir="ltr"><span>Parallel-readdir</span></p>
</td>
<td>
<p dir="ltr"><span>Proposed Solution</span></p>
</td>
</tr>
<tr>
<td>
<p dir="ltr"><span>FS Crawl Time in Seconds</span></p>
</td>
<td>
<p dir="ltr"><span>16497.523</span></p>
</td>
<td>
<p dir="ltr"><span>8717.872</span></p>
</td>
<td>
<p dir="ltr"><span>4261.401 </span></p>
</td>
</tr>
</tbody>
</table>
</div>
<p>In short, the basic idea behind the proposal is the efficient
managing of readdir buffer in gluster along with prefetching
the dentries for intelligent switch-over to the next buffer.
The detailed problem description, deign description and
results are available in the doc.<a href="https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit" target="_blank">https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit</a>
<br>
</p>
<p><br>
</p>
<p>If anybody can help with the testing on a different kind of
workloads, I would be very happy to assist. If wanted to test
the patch and run a performance test on your setup, I could
help with back-porting the patch to the version of your
choice.<br>
</p>
<p><br>
</p>
<p><a href="https://review.gluster.org/24469" target="_blank">https://review.gluster.org/24469</a></p>
<p><a href="https://review.gluster.org/24470" target="_blank">https://review.gluster.org/24470</a><br>
</p>
<p><br>
</p>
<p>Regards</p>
<p>Rafi KC<br>
</p>
<br>
<fieldset></fieldset>
<pre>________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: <a href="https://bluejeans.com/441850968" target="_blank">https://bluejeans.com/441850968</a>
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a>
</pre>
</blockquote>
<br>
<fieldset></fieldset>
<pre>________
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: <a href="https://bluejeans.com/441850968" target="_blank">https://bluejeans.com/441850968</a>
Gluster-users mailing list
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a>
</pre>
</blockquote>
</div>
________<br>
<br>
<br>
<br>
Community Meeting Calendar:<br>
<br>
Schedule -<br>
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC<br>
Bridge: <a href="https://bluejeans.com/441850968" rel="noreferrer" target="_blank">https://bluejeans.com/441850968</a><br>
<br>
Gluster-users mailing list<br>
<a href="mailto:Gluster-users@gluster.org" target="_blank">Gluster-users@gluster.org</a><br>
<a href="https://lists.gluster.org/mailman/listinfo/gluster-users" rel="noreferrer" target="_blank">https://lists.gluster.org/mailman/listinfo/gluster-users</a><br>
</blockquote></div></div>