[Gluster-devel] Multithreaded Iterative Dir Tree Scan

Amar Tumballi amar at kadalu.io
Fri Apr 24 02:35:20 UTC 2020


This looks like a good effort to pick up Barak. A needed one indeed.

-Amar

On Mon, Mar 23, 2020 at 3:18 PM Barak Sason Rofman <bsasonro at redhat.com>
wrote:

> Hello everyone!
> Following a discussion I had with @Susant Palai some time ago, we have
> decided to look into an option to improve the rebalance process in the DHT
> layer by modifying the underlying mechanism. Currently, dir-tree crawling
> is done recursively, by a single thread, which is likely slow and also
> poses the risk of stack overflow. An iterative multithreaded solution might
> improve performance and also stability (by eliminating the risk of stack
> overflow). I have prepared a POC doc on the matter, including a sample
> implementation of the iterative multithreaded solution. The doc can be
> found at:
>
> https://docs.google.com/document/d/1JCl0T9zeagOcFFpgVQF8zNyhlR54VqkNAZ7TJb42egE/edit
>
> <https://docs.google.com/document/d/1L0uHgFbrNWWxCQB6s4YcoymKrO7q0yVAbEIWWIiu_as/edit?usp=sharing>Apart
> from the rebalance process, maybe this approach can be useful for other
> use-cases where dir-tree crawl is being performed? Any comments on the
> concept, the design of the solution and the implementation are welcome.
>
> --
> *Barak Sason Rofman*
>
> Gluster Storage Development
>
> Red Hat Israel <https://www.redhat.com/>
>
> 34 Jerusalem rd. Ra'anana, 43501
>
> bsasonro at redhat.com <adi at redhat.com>    T: *+972-9-7692304*
> M: *+972-52-4326355*
> <https://red.ht/sig>
> _______________________________________________
>
> Community Meeting Calendar:
>
> Schedule -
> Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
> Bridge: https://bluejeans.com/441850968
>
>
>
>
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> https://lists.gluster.org/mailman/listinfo/gluster-devel
>
>

-- 
--
https://kadalu.io
Container Storage made easy!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20200424/1b76195d/attachment.html>


More information about the Gluster-devel mailing list