[Gluster-devel] RPC thoughts for glusterd-2.0

[Kaushal M]

Resending to mailing list.

---------- Forwarded message ----------
From: "Kaushal M" <kshlmster at gmail.com>
Date: 03-Aug-2015 10:12
Subject: Re: [Gluster-devel] RPC thoughts for glusterd-2.0
To: "chris holcombe" <chris.holcombe at canonical.com>
Cc: "Krishnan Parthasarathi" <kparthas at redhat.com>

> Hi Chris,
>
> In GlusterD-2.0, all the gluster command endpoints will be RESTful
> http api. The gluster cli itself will be a REST client. This should
> allow for easier and better integration for external consumers. We've
> not yet fleshed out the api for all the commands yet. We've started
> off with Heketi's api, which already has it designed for volume
> creation, deletion and listing, and for cluster growth.
>
> While the REST api is useful for external consumers, we think it's not
> the most suitable method for doing internal communication within the
> cluster. This is the communication between GlusterD and other
> GlusterDs, bricks, clients etc. We are looking for a good RPC
> framework with libraries for Go and C. There are not a lot around.
> We've looked at 3 so far.
>
> 1. gRPC - This is really interesting and provides a lot of features.
> But this is currently in alpha and also requires the use of protobuf3,
> which itself is in alpha.
> 2. Apache Thrift - The C implementation requires the use of Glib,
> which we are not comfortable with and feel is a little too much for
> what we need.
> 3. Json-rpc - This works wonderfully well with Go. But requires a lot
> of manual boiler-plate code writing for C to do the serialization and
> deserialization.
>
> We could implement our own sunrpc for Go using go-xdr, but one
> drawback is that, we don't have code generation for Go from .x files.
> This requires us to manually write the request and response structs.
>
> We're currently looking at writing our own simple RPC framework using
> protobuf2 for data serialization. Protobuf has a couple of advantages
> over XDR. It's got code-generators for Go and C, and protobuf
> serialized structs are forwards and backwards compatible. Either KP or
> I will be sharing details regarding this soon.
>
> If you've got any suggestion, please let us know.
>
> And thanks for showing interest in GlusterD-2.0!
>
> Cheers,
> Kaushal
>
>
> On Fri, Jul 31, 2015 at 11:13 PM, Atin Mukherjee
> <atin.mukherjee83 at gmail.com> wrote:
> > -Atin
> > Sent from one plus one
> > On Jul 31, 2015 10:57 PM, "chris holcombe" <chris.holcombe at canonical.com
>
> > wrote:
> >>
> >> Thanks!  I'd like to contribute if I can.  I found a go xdr library:
> >> https://github.com/davecgh/go-xdr
> >> That should be enough to get the sun rpc stuff working.  :)
> > +KP,kaushal - this may reduce our effort?
> >
> >
> >>
> >>
> >> On 07/31/2015 10:26 AM, Atin Mukherjee wrote:
> >>>
> >>> Thanks Chris for your mail.
> >>>
> >>> We've started exploring different RPC techniques as we plan to write
> >>> glusterd 2.0 in go and it seems like go doesn't have sun RPC library
> >>> support. All the details will be put up once we share the glusterd
2.0 high
> >>> level plan.
> >>>
> >>> Having said that, we'd definitely consider your points in terms of
> >>> designing it. Also any form of contribution for glusterd 2.0 Is more
than
> >>> welcome :)
> >>>
> >>> -Atin
> >>> Sent from one plus one
> >>>
> >>> On Jul 31, 2015 10:43 PM, "chris holcombe" <
chris.holcombe at canonical.com>
> >>> wrote:
> >>>>
> >>>> I've been working on writing a gluster library that speaks the native
> >>>> RPC to gluster so that I could avoid using the CLI.  I've been
successful
> >>>> but the endeavor has brought up some interesting insights.  I plan
on open
> >>>> sourcing the code so that everyone can benefit from this.
> >>>>
> >>>> struct gf_cli_req {
> >>>>         opaque  dict<>;
> >>>> };
> >>>> and
> >>>> struct gf_cli_rsp {
> >>>>         int     op_ret;
> >>>>         int     op_errno;
> >>>>         string  op_errstr<>;
> >>>>         opaque  dict<>;
> >>>> };
> >>>>
> >>>> are used to make a lot of the CLI calls. The opaque dict passing back
> >>>> and forth creates a need for string parsing on both sides.  My
understanding
> >>>> of RPC was that each call would have its own struct associated with
it so
> >>>> that no parsing need happen.  It would also make it much easier for
other
> >>>> people to build libraries to interface with Gluster if the call
parameters
> >>>> were clearly defined.  Each time I want to make a call work with
glusterd
> >>>> 1.0 I need to figure out what parameters need to be packed into the
dict.
> >>>> This involves some hunting.
> >>>>
> >>>> Since glusterd 2.0 is a refactoring I figured this would be a good
time
> >>>> to ask for adding additional more explicit RPC calls to the API.
That would
> >>>> make it a breeze to run an rpcgen command against the .x file and
have a
> >>>> working library to glusterd!
> >>>>
> >>>> Thanks,
> >>>> Chris
> >>>> _______________________________________________
> >>>> Gluster-devel mailing list
> >>>> Gluster-devel at gluster.org
> >>>> http://www.gluster.org/mailman/listinfo/gluster-devel
> >>
> >>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150804/5f118e58/attachment.html>