[Gluster-devel] Possible improvements to Gluster epoll-based event layer

Krishnan Parthasarathi kparthas at redhat.com
Wed Jul 8 13:10:20 UTC 2015


The following is a list of things that come to my mind
whenever I (re)visit our epoll-based event implementation
and compare it with how other open-source server projects
use epoll(7). The following list assumes familiarity with
epoll(7) and how we use it. Feel free to email gluster-devel with
your question. I will answer them to the best of my knowledge.

GlusterFS epoll subsystem

    GlusterFS uses epoll(7) on GNU/Linux systems for receiving notifications on

Current design

    We use a single epoll fd, which is being 'waited' by a configurable no. of
    threads (via {client,server}.event-threads option). All sockets are configured
    to be NONBLOCKing and added to this epoll fd. We use level-triggered mode with

Open issues

    0. Find a way to profile GlusterFS' epoll-based event system.

        a. to measure its contribution to latency of a FOP, IOPs. 
        b. to measure lock contention among `worker` threads.

    1. Move to edge-triggered mode for benefits mentioned in epoll(7)

    2. Increase `maxevents` handled per `epoll_wait(2)`. This helps in batching
       notifications with one system call.


More information about the Gluster-devel mailing list