[Gluster-users] Readdirp (ls -l) Performance Improvement

[Felix Kölzow]

Dear Rafi,


thanks for your effort. I think this is of great interest of many
gluster users. Thus, I would really encourage you to

test and to further improve this feature. Maybe it is beneficial to
create a certain guideline which things should be tested

to make this feature really ready for productive use.


Thanks in advance.

Felix

On 27/05/2020 07:56, RAFI KC wrote:
>
> Hi All,
>
> I have been working on POC to improve readdirp performance
> improvement. At the end of the experiment, The results are showing
> promising result in performance, overall there is a 104% improvement
> for full filesystem crawl compared to the existing solution. Here is
> the short test numbers. The tests were carried out in 16*3 setup with
> 1.5 Million dentries (Both files and dir). The system also contains
> some empty directories. *In the result the proposed solution is 287%
> faster than the plane volume and 104% faster than the parallel-readdir
> based solution.*
>
> *
> *
>
> Configuration
>
>
>
> Plain volume
>
>
>
> Parallel-readdir
>
>
>
> Proposed Solution
>
> FS Crawl Time in Seconds
>
>
>
> 16497.523
>
>
>
> 8717.872
>
>
>
> 4261.401
>
> In short, the basic idea behind the proposal is the efficient managing
> of readdir buffer in gluster along with prefetching the dentries for
> intelligent switch-over to the next buffer. The detailed problem
> description, deign description and results are available in the
> doc.https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit
>
>
>
> If anybody can help with the testing on a different kind of workloads,
> I would be very happy to assist. If wanted to test the patch and run a
> performance test on your setup, I could help with back-porting the
> patch to the version of your choice.
>
>
> https://review.gluster.org/24469
>
> https://review.gluster.org/24470
>
>
> Regards
>
> Rafi KC
>
>
> ________
>
>
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
>
> Gluster-users mailing list
> Gluster-users at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200527/6cb248b2/attachment.html>