[Gluster-devel] Introducing an eventing framework for GlusterFS through storaged

[Samikshan Bairagya]

Hi everyone,

I have been working on this project for the past few weeks that aims at 
improving the eventing framework for GlusterFS through storaged [0][1]. 
Through a DBus API over the existing GlusterFS CLI, storaged could help 
with better notifications for gluster events.

The plan:
=========

storaged exports objects implementing the respective interfaces for 
Linux block devices, drives, RAID arrays, etc. on the DBus. More objects 
implementing other interfaces can be exported through modules. The 
"glusterfs" module in storaged will populate the DBus with GlusterFS 
specific objects implementing the required interfaces. As an example, 
the "iscsi" module in storaged adds DBus objects implementing the 
org.storaged.Storaged.ISCSI.Session interface[2].

Once DBus objects are exported to the DBus for GlusterFS volumes and 
bricks, implementing the respective interfaces, it would be convenient 
for interested clients to receive event notifications through DBus 
signals or method calls. This enables clients to get updates wrt changes 
on the glusterfs side in real time over the existing logging framework.

The implementation can be divided into 2 major steps:

1. Retrieving the GlusterFS state and populating the bus accordingly
--------------------------------------------------------------------

All volumes and bricks need to be exported as objects on the bus after 
retrieving the glusterfs state. The volumes objects would be under the 
object path /org/storaged/Storaged/glusterfs/volume/* and would be 
implementing the org.storaged.Storaged.GlusterFS.Volume interface. 
Similarly, bricks objects implementing the corresponding 
o.s.S.GlusterFS.Brick should appear under path /o/s/S/glusterfs/brick/*.

2. Update storaged with GlusterFS state and send change notifications
---------------------------------------------------------------------

This step is important wrt event notifications. The exported DBus 
objects implementing the corresponding interfaces hold properties. When 
changes in the GlusterFS state are recorded in storaged, new objects 
will be added, interfaces will be added to existing objects and/or 
properties will be updated. These changes will send out corresponding 
DBus signals. Clients registered on these signals can receive change 
notifications accordingly.

Working prototype for the "glusterfs" module in storaged
========================================================

I have added a "glusterfs" module to storaged which can be considered to 
be a very basic prototype to give an overview of how this can work.

Retrieving the glusterfs state
------------------------------

The prototype retrieves created GlusterFS volumes by parsing the output 
of "gluster volume info all --xml"  and exports them as DBus objects 
under /org/storaged/Storaged/glusterfs/volume/*. Right now the prototype 
implementation contains only the volume name as one of the properties 
for the objects.


Updating storaged and sending out change notifications
------------------------------------------------------

storaged responds to a DBus method call made by a script under 
/var/lib/glusterd/hooks/1/create/post/S01StoragedReload.sh every time a 
new volume is created by "reloading" the glusterfs module. This adds a 
new object corresponding to the newly created volume. "InterfacesAdded" 
signal emitted by the object at "/org/storaged/Storaged" can be caught 
to be notified of such events.

Testing the prototype:
======================

1. Obtain the source and checkout the "glusterfs" branch from 
https://github.com/samikshan/storaged.git

2. Install storaged:

$ ./auotogen.sh
$ ./configure --enable-glusterfs --prefix=/usr --sysconfdir=/etc 
--localstatedir=/var
$ make
$ sudo make install

3. Add the script 
https://github.com/samikshan/storaged/blob/glusterfs/modules/glusterfs/data/S01StoragedReload.sh 
under /var/lib/glusterd/hooks/1/create/post/

4. Create one or more gluster volumes and start storaged in debug mode 
by issuing the following command:

sudo /usr/libexec/storaged/storaged -r --force-load-modules

5. Use gdbus/qdbus to introspect the org.storaged.Storaged bus 
connection. The gluster volumes should appear as DBus objects under the 
object path /org/storaged/Storaged/glusterfs/volume/*

6. Create a new gluster volume and introspect the org.storaged.Storaged 
connection again. A new DBus object corresponding to the new volume 
created should now appear.

This prototype will hopefully extend itself to a full-fledged eventing 
framework for glusterfs. If you have any feedback or queries on the 
design or prototype please ask.

[0] https://storaged-project.github.io/
[1] https://storaged-project.github.io/doc/latest/index.html
[2] 
http://storaged-project.github.io/doc/latest/gdbus-org.storaged.Storaged.ISCSI.Session.html

Thanks and Regards,

Samikshan Bairagya
Software Engineer,
Storage Engineering,
Red Hat India Pvt. Ltd.