[Gluster-devel] RPC help

[Raghavendra G]

On Sat, Jul 25, 2015 at 5:12 AM, chris holcombe <
chris.holcombe at canonical.com> wrote:

> Hey Everyone!
>
> I'm having trouble figuring out how the RPC mechanism works in Gluster.
> Any help would be hugely appreciated!
> When a client makes a request from the glusterd service an xdr encoded rpc
> is sent to it.  From what I'm gathering by looking at the source and GDB
> the main struct to be concerned with is:
>
> struct rpc_transport_req {
>         rpc_transport_msg_t  msg;
>         rpc_transport_rsp_t  rsp;
>         struct rpc_req      *rpc_req;
> };
>
> From tracing it down to writev I see that the data looks as follows before
> being written out:
>
> (gdb) p req
> $4 = {msg = {rpchdr = 0x7ffff36c7c00, rpchdrcount = 1, proghdr =
> 0x7ffff36c7d20,
>     proghdrcount = 1, progpayload = 0x0, progpayloadcount = 0, iobref =
> 0x7fffe4000920}, rsp = {
>     rsphdr = 0x0, rsphdr_count = 0, rsp_payload = 0x0, rsp_payload_count =
> 0, rsp_iobref = 0x0},
>   rpc_req = 0x555555802dac}
>
> I'm having trouble figuring out if this req variable is actually what is
> being written out to the wire.
>
> Now on the reply side the first header is the length of the reply. That
> makes sense.  What I'm not getting is if the rpc_transport_req is actually
> being serialized with xdr or are parts of it being serialized?  It's hard
> to tell.  I need to read the __socket_proto_state_machine over again and
> understand it better.  I searched around for notes or anything online and
> there's really almost zero written about this.
>
> So my questions are:
> 1. Is the rpc_transport_req being xdr encoded onto the wire?
>

No, its the contents of rpc_transport_msg_t (req->msg, defined in
rpc-transport.h) which is written onto the wire. For more details look into
functions __socket_ioq_new and __socket_ioq_churn_entry (both defined in
./rpc/rpc-transport/socket/src/socket.c).

2. Are there any good materials out there that explain the protocol?
> Figuring it out by reading the source and break pointing the code is slow
> going.  Esp with the epoll threads making tracking things down hard.
>

Our rpc implementation is based on RFC 5531 [1]. To get an high level flow,
you can look at socket_submit_request and socket_submit_reply and keep
following the control flow.

[1] https://tools.ietf.org/html/rfc5531

regards,
Raghavendra.


> Thanks for the help!
> -Chris
> _______________________________________________
> Gluster-devel mailing list
> Gluster-devel at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-devel
>



-- 
Raghavendra G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150727/8cac890c/attachment.html>