[Gluster-devel] GlusterD 2.0 2-3 months plan

[Nagaprasad Sathyanarayana]

On 08/04/2015 02:57 PM, Krishnan Parthasarathi wrote:
> # GlusterD 2.0 plan (Aug-Oct '15)
>
> [This text in this email structured using markdown format]
>
> This document outlines efforts planned towards [thousand-node-glusterd][1] in
> the coming 2-3 months. The following email thread on gluster-devel provides
> context on previous discussions around glusterd-2.0 -
> http://www.gluster.org/pipermail/gluster-users/2014-September/018639.html
>
> ## High-level tasks
>
> 1. Define a repeatable environment using Vagrant+ansible.
>
>      - Encourages early adoption.
>
> 1. Introduction of etcd/consul to store configuration information.
>
>      - Define deployment steps where applicable, targeting existing
>        gluster-users.
>
>      - Add admin guide doc.
>
> 1. Build a transaction framework to support implementation of gluster commands.
>
>      - Define interfaces for commands to integrate with the transaction
>        framework.
>
>      - Add developer doc for porting commands into this framework - `Porting Guide`.
>
> 1. Implement basic volume management and peer commands.
>
>      - e.g, volume-create, volume-start, volume-add-brick, volume-remove-brick,
>        volume-stop, volume-delete, peer-probe, peer-detach and peer-status.
>
>      - Add some of these to the `Porting Guide`.
>
> ## Progress on current activities
>
> 1. Setting up codebase
>      - Has consul as the default configuration store
>
>      - Uses [libkv][2] library to interface our store. Makes it possible to move to
>        other options if required.
>
>
>      - Begun implementing REST endpoints defined by [heketi][3].
>
>      - Begun implementing volume file generation in Go; This support existing
>        volume configurations only. Other volume file generation proposals are
>        out of the current scope.
>
> 2. Cross language RPC framework
>      - Exploring the following [options][4]
>
>          - gRPC - This is really interesting and provides a lot of features.
>            But this is currently in alpha and also requires the use of
>            protobuf3, which itself is in alpha.
>
>          - Apache Thrift - The C implementation requires the use of Glib,
>            which we are not comfortable with and feel is a little too much for
>            what we need.
>
>          - JSON-RPC - This works wonderfully well with Go. But requires a lot
>            of manual boiler-plate code writing for C to do the serialization and
>            deserialization.
>
>      - Properties that we are looking for in frameworks
>
>          - Code generation for serialization
>
>          - Optional stub code generation for the RPC services themselves
>
>          - Ease of programming in the framework
In case this is not already looked into, a good comparative study 
between Avro, Thrift and Protocol buffer can be found here: 
http://www.slideshare.net/IgorAnishchenko/pb-vs-thrift-vs-avro.

>
> ## Activities lined up
>
> 1.  Setting up development environment
>
>      - We choose to use Vagrant+Ansible.
>
> 2.  Settle down on a cross language RPC framework
>
>      - This also involves getting RPC handlers in glusterfs codebase.
>
> 3.  Design/Prototype a transaction framework
>
>      - Aim to make integrating new features easier than in current glusterd.
>
>      - Should make porting of existing commands simple.
>
> [1]: http://www.gluster.org/community/documentation/index.php/Features/thousand-node-glusterd
> [2]: https://github.com/docker/libkv
> [3]: https://github.com/heketi/heketi/wiki/API
> [4]: http://www.gluster.org/pipermail/gluster-devel/2015-August/046307.html
>
> ~GlusterD team
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>
>