[Gluster-devel] Some updates on the eventing framework for Gluster

[Nagaprasad Sathyanarayana]

Any specific reasons for going with Kafka? What is the advantage of using Kafka over RabbitMQ? 

Thanks
Naga


> On Dec 2, 2015, at 6:09 AM, Samikshan Bairagya <@redhat.com> wrote:
> 
> Hi,
> 
> The updates for the eventing framework for gluster can be divided into the following two parts.
> 
> 1. Bubbling out notifications through dbus signals from every gluster node.
> 
> * The 'glusterfs' module in storaged [1] exports objects on the system bus for every gluster volume. These objects hold the following properties:
> - Name
> - Id
> - Status (0 = Created, 1 = Started, 2 = Stopped)
> - Brickcount
> 
> * A singleton dbus object corresponding to glusterd is also exported by storaged on the system bus. This object holds properties to track the state of glusterd (LoadState and ActiveState).
> 
> 2. Aggregating all these signals from each node over an entire cluster.
> 
> * Using Kafka [2] for messaging over a cluster: Implementing a (dbus signal) listener in python that converts these dbus signals from objects to 'keyed messages' in Kafka under a particular 'topic'.
> 
> For example, if a volume 'testvol' is started, a message is published under topic 'testvol', with 'status' as the 'key' and the changed status ('1' in this case) as the 'value'.
> 
> 
> *** Near term plans:
> - Export dbus objects corresponding to bricks.
> - Figure out how to map the path to the brick directory to the block device and consequently the drive object. The 'SmartFailing' property from org.storaged.Storaged.Drive.Ata [3] interface can then be used to track brick failures.
> - Make the framework work over a multi-node cluster with possibly a multi-broker kafka setup to identify redundancies as well as to keep consistent information across the cluster.
> 
> Views/feedback/queries are welcome.
> 
> [1] https://github.com/samikshan/storaged/tree/glusterfs
> [2] http://kafka.apache.org/documentation.html#introduction
> [3] http://storaged-project.github.io/doc/latest/gdbus-org.storaged.Storaged.Drive.Ata.html#gdbus-property-org-storaged-Storaged-Drive-Ata.SmartFailing
> 
> Thanks and Regards,
> 
> Samikshan
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
> 
>