[Gluster-devel] I/O performance

[Xavi Hernandez]

Hi,

I've been doing some tests with the global thread pool [1], and I've
observed one important thing:

Since this new thread pool has very low contention (apparently), it exposes
other problems when the number of threads grows. What I've seen is that
some workloads use all available threads on bricks to do I/O, causing
avgload to grow rapidly and saturating the machine (or it seems so), which
really makes everything slower. Reducing the maximum number of threads
improves performance actually. Other workloads, though, do little I/O
(probably most is locking or smallfile operations). In this case limiting
the number of threads to a small value causes a performance reduction. To
increase performance we need more threads.

So this is making me thing that maybe we should implement some sort of I/O
queue with a maximum I/O depth for each brick (or disk if bricks share same
disk). This way we can limit the amount of requests physically accessing
the underlying FS concurrently, without actually limiting the number of
threads that can be doing other things on each brick. I think this could
improve performance.

Maybe this approach could also be useful in client side, but I think it's
not so critical there.

What do you think ?

Xavi

[1] https://review.gluster.org/c/glusterfs/+/20636
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.gluster.org/pipermail/gluster-devel/attachments/20190131/e46c9b0d/attachment.html>