[Gluster-devel] Shared resource pool for libgfapi

[Krishnan Parthasarathi]

> Every resource(thread, mem pools) is associated with glusterfs_ctx, hence as
> the ctxs in the process
> grows the resource utilization also grows (most of it being unused). This
> mostly is an issue with any
> libgfapi application: USS, NFS Ganesha, Samba, vdsm, qemu.
> It is normal in any of the libgfapi application to have multiple mounts(ctxs)
> in the same process,
> we have seen the number of threads scale from 10s-100s in these applications.
> 
> Solution:
> ======
> Have a shared resource pool, threads and mem pools. Since they are shared
> have a scaling policy that
> scales based on the number of ctxs.
> 
> Resources that can be shared:
> - Synctask threads
> - Timer threads, circular buf timer threads
> - Sigwaiter thread
> - poller threads, these aren't as straight forward as others. If we want to
> share the poll threads,
> we will have to reuse ports which is a different topic. Hence keeping poller
> threads out of this mail as of now.

event_pool can also be shared. event_pool isn't aware of port et al. It merely
uses multiple threads to handle events like EPOLLIN, EPOLLOUT and EPOLLERR on
all the socket fds that are registered. These socket fds may belong to different
fs objects in the same process.