[Gluster-devel] Multithreaded Iterative Dir Tree Scan
sankarshan.mukhopadhyay at gmail.com
Fri Apr 24 04:13:03 UTC 2020
On Fri, 24 Apr 2020 at 08:05, Amar Tumballi <amar at kadalu.io> wrote:
> This looks like a good effort to pick up Barak. A needed one indeed.
Should this be tracked with a release label and planned? The content of the
document should probably transfer itself to the issue tracking the PR(s)
> On Mon, Mar 23, 2020 at 3:18 PM Barak Sason Rofman <bsasonro at redhat.com>
>> Hello everyone!
>> Following a discussion I had with @Susant Palai some time ago, we have
>> decided to look into an option to improve the rebalance process in the DHT
>> layer by modifying the underlying mechanism. Currently, dir-tree crawling
>> is done recursively, by a single thread, which is likely slow and also
>> poses the risk of stack overflow. An iterative multithreaded solution might
>> improve performance and also stability (by eliminating the risk of stack
>> overflow). I have prepared a POC doc on the matter, including a sample
>> implementation of the iterative multithreaded solution. The doc can be
>> found at:
>> from the rebalance process, maybe this approach can be useful for other
>> use-cases where dir-tree crawl is being performed? Any comments on the
>> concept, the design of the solution and the implementation are welcome.
>> *Barak Sason Rofman*
>> Gluster Storage Development
>> Red Hat Israel <https://www.redhat.com/>
>> 34 Jerusalem rd. Ra'anana, 43501
>> bsasonro at redhat.com <adi at redhat.com> T: *+972-9-7692304*
>> M: *+972-52-4326355*
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Gluster-devel