[Gluster-devel] Regarding multi-threaded epoll and notify

[Pranith Kumar Karampuri]

hi,
         With the addition of multi-threaded epoll functionality, 
handling CHILD_UP/CHILD_DOWN in cluster xlators gets tricky. i.e. Let us 
assume one of the bricks in replication is down while the other one is 
up. Now if in parallel the brick that went down comes up and the one 
that is up goes down, there is a possibility for wrong CHILD_UP/DOWN to 
go to parent of afr, similar for dht as well. One way to fix it is each 
cluster xlator handle this correctly but it entails calling 'notify' 
inside mutex locks to the parent translator. Another way to fix it is 
that we should add the CHILD_UP/CHILD_DOWN related epoll events from rpc 
layer into a queue and have a dedicated thread process these events, 
something similar to how timer thread processes events. If this gets in 
we get best of both worlds where CHILD_UP/DOWN will be single threaded 
so xlators don't have to worry about it, while the fops get 
multi-threaded behaviour.

I like the second approach. What do you guys suggest?

Pranith