[Gluster-devel] GlusterD 2.0 2-3 months plan

[Krishnan Parthasarathi]

# GlusterD 2.0 plan (Aug-Oct '15)

[This text in this email structured using markdown format]

This document outlines efforts planned towards [thousand-node-glusterd][1] in
the coming 2-3 months. The following email thread on gluster-devel provides
context on previous discussions around glusterd-2.0 -
http://www.gluster.org/pipermail/gluster-users/2014-September/018639.html

## High-level tasks

1. Define a repeatable environment using Vagrant+ansible.

    - Encourages early adoption.

1. Introduction of etcd/consul to store configuration information.

    - Define deployment steps where applicable, targeting existing
      gluster-users.

    - Add admin guide doc.

1. Build a transaction framework to support implementation of gluster commands.

    - Define interfaces for commands to integrate with the transaction
      framework.

    - Add developer doc for porting commands into this framework - `Porting Guide`.

1. Implement basic volume management and peer commands.

    - e.g, volume-create, volume-start, volume-add-brick, volume-remove-brick,
      volume-stop, volume-delete, peer-probe, peer-detach and peer-status.

    - Add some of these to the `Porting Guide`.

## Progress on current activities

1. Setting up codebase
    - Has consul as the default configuration store

    - Uses [libkv][2] library to interface our store. Makes it possible to move to
      other options if required.


    - Begun implementing REST endpoints defined by [heketi][3].

    - Begun implementing volume file generation in Go; This support existing
      volume configurations only. Other volume file generation proposals are
      out of the current scope.

2. Cross language RPC framework
    - Exploring the following [options][4]

        - gRPC - This is really interesting and provides a lot of features.
          But this is currently in alpha and also requires the use of
          protobuf3, which itself is in alpha.

        - Apache Thrift - The C implementation requires the use of Glib,
          which we are not comfortable with and feel is a little too much for
          what we need.

        - JSON-RPC - This works wonderfully well with Go. But requires a lot
          of manual boiler-plate code writing for C to do the serialization and
          deserialization.

    - Properties that we are looking for in frameworks

        - Code generation for serialization

        - Optional stub code generation for the RPC services themselves

        - Ease of programming in the framework

## Activities lined up

1.  Setting up development environment

    - We choose to use Vagrant+Ansible.

2.  Settle down on a cross language RPC framework

    - This also involves getting RPC handlers in glusterfs codebase.

3.  Design/Prototype a transaction framework

    - Aim to make integrating new features easier than in current glusterd.

    - Should make porting of existing commands simple.

[1]: http://www.gluster.org/community/documentation/index.php/Features/thousand-node-glusterd
[2]: https://github.com/docker/libkv
[3]: https://github.com/heketi/heketi/wiki/API
[4]: http://www.gluster.org/pipermail/gluster-devel/2015-August/046307.html

~GlusterD team