[Gluster-users] Readdirp (ls -l) Performance Improvement

[RAFI KC]

Hi All,

I have been working on POC to improve readdirp performance improvement. 
At the end of the experiment, The results are showing promising result 
in performance, overall there is a 104% improvement for full filesystem 
crawl compared to the existing solution. Here is the short test numbers. 
The tests were carried out in 16*3 setup with 1.5 Million dentries (Both 
files and dir). The system also contains some empty directories. *In the 
result the proposed solution is 287% faster than the plane volume and 
104% faster than the parallel-readdir based solution.*

*
*

Configuration

	

Plain volume

	

Parallel-readdir

	

Proposed Solution

FS Crawl Time in Seconds

	

16497.523

	

8717.872

	

4261.401

In short, the basic idea behind the proposal is the efficient managing 
of readdir buffer in gluster along with prefetching the dentries for 
intelligent switch-over to the next buffer. The detailed problem 
description, deign description and results are available in the 
doc.https://docs.google.com/document/d/10z4T5Sd_-wCFrmDrzyQtlWOGLang1_g17wO8VUxSiJ8/edit 



If anybody can help with the testing on a different kind of workloads, I 
would be very happy to assist. If wanted to test the patch and run a 
performance test on your setup, I could help with back-porting the patch 
to the version of your choice.


https://review.gluster.org/24469

https://review.gluster.org/24470


Regards

Rafi KC

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-users/attachments/20200527/596c6ffd/attachment.html>