[Gluster-devel] Updates on GD2

[Kaushal M]

(I had to send this out a long time ago. But better late than never).

Hi all,

We (GD2 developers) had a discussion with Jeff when he was down in
Bangalore late last month.
We discussed mainly on 3 topics, and these are my notes from these meetings.

## GD2 volgen

This was a carry forward of the meeting I had at the developer
gatherings in Brno.
The minutes and update from that meeting are available at [1][1].
We continued this meeting by quickly recounting the Brno meeting.

We discussed the systemd-unit style approach for volgen.
It was agreed that this approach seemed suitable for a
flexible/exetensible volgen.

We also discussed if it would be possible to include a filters sort of
a feature with in this approach.
Xlators could specify a filter/script in their config files.
This script would help in doing more complex operations on the xlator,
which cannot be done by GD2 with just the information from the config file.
This feature could be used for example to generate asymmetric graphs,
like the ones generated when user serviceable snapshots are enabled.

We'll write up a proper design proposal for the volgen and send it for comments.


## GD2 & ETCD

We discussed the progress being done with the ETCD integration/bootstrapping.

Atin and Gaurav have been working on this, following the design
proposed by Jeff in [2][2].

We also discussed the need to allow users to use their own/existing
ETCD clusters,
instead of the cluster GD2 creates.

We also found out that we haven't properly planned (or even
investigated) how TLS can be used with ETCD.
We plan to do this investigation soon, and follow that up with
planning how we can manage a TLS ETCD cluster.

## GD2 plugins

In the discussion we tried to refine the requirements for a GD2 plugin.
We concluded the following.

Plugins are expected for the following in GD2,

1. ReST - add and handle new rest endpoints
2. CLI - add new commands to CLI, more often would involve
3. Services - allow GD2 to start and manage custom processes.

Plugin interfaces for each type will be defined.
This will allow, for eg., a feature like geo-replication to integrate
with GD2 without writing any actual GD2 code.

To help plugins work well, GD2 will also provide a consumer API.
Plugins will have access to consumer API for the follwing,

- store access
- daemon management
- volfile regeneration
- transaction framework
- all ReST endpoints to perform other commands

These are just the expected Plugin interfaces and consumer APIs right now.
More interfaces and APIs may be added as we start implementing actual plugins.

In addition to the above, we discussed multi-language support.
The following points were agreed on,

- The plugin protocol will be properly defined and documented
 - This document can be used to implement the plugin interface and
write the plugins in any language
- Official language support will involve supplying and maintaining
plugin libraries for the selected languages.
 - Go plugin packages will be provided
 - C & Python could also be supported
- Official plugins (plugins shipped with GlusterFS) will ideally be in Go

We also discussed an existing Go plugin package, [`hashicorp/go-plugin`][3],
as a possible candidate for implementing GD2 plugins.

Plugins in Go have to be written as standalone processes,
which communicate with the host process over RPC.
`hashicorp/go-plugin` is a package that implements almost everything
required for this type of plugins.

The `hashicorp/go-plugin` has the following positives and negatives.

Positives,
- Stable and well tested. Used in large existing Hashicorp projects.
- Good documentation
- Easy to use
- New features like signed plugins on the way.
- Protocol uses msgpack encoding, good for multi lanuguage support

Negatives,
- Custom connection multiplexer, [hashicorp/yamux][4]
 - Go only implementation right now
 - Implementations not available for other language
 - Spec is available for reference
- Go only

We didn't come to a decision on if we will be using
`hashicorp/go-plugin` in GD2.
We'd like to know the opinions of the others on this.

I think I've covered everything discussed in the meeting with Jeff.
If I've missed anything please let me know.

Thanks,
Kaushal


[1]: https://public.pad.fsfe.org/p/gluster-gd2-kaushal
[2]: https://www.gluster.org/pipermail/gluster-devel/2015-September/046740.html
[3]: https://github.com/hashicorp/go-plugin
[4]: https://github.com/hashicorp/yamux