[Gluster-devel] RPC help

[chris holcombe]

Hey Everyone!

I'm having trouble figuring out how the RPC mechanism works in Gluster.  
Any help would be hugely appreciated!
When a client makes a request from the glusterd service an xdr encoded 
rpc is sent to it.  From what I'm gathering by looking at the source and 
GDB the main struct to be concerned with is:

struct rpc_transport_req {
         rpc_transport_msg_t  msg;
         rpc_transport_rsp_t  rsp;
         struct rpc_req      *rpc_req;
};

 From tracing it down to writev I see that the data looks as follows 
before being written out:

(gdb) p req
$4 = {msg = {rpchdr = 0x7ffff36c7c00, rpchdrcount = 1, proghdr = 
0x7ffff36c7d20,
     proghdrcount = 1, progpayload = 0x0, progpayloadcount = 0, iobref = 
0x7fffe4000920}, rsp = {
     rsphdr = 0x0, rsphdr_count = 0, rsp_payload = 0x0, 
rsp_payload_count = 0, rsp_iobref = 0x0},
   rpc_req = 0x555555802dac}

I'm having trouble figuring out if this req variable is actually what is 
being written out to the wire.

Now on the reply side the first header is the length of the reply. That 
makes sense.  What I'm not getting is if the rpc_transport_req is 
actually being serialized with xdr or are parts of it being serialized?  
It's hard to tell.  I need to read the __socket_proto_state_machine over 
again and understand it better.  I searched around for notes or anything 
online and there's really almost zero written about this.

So my questions are:
1. Is the rpc_transport_req being xdr encoded onto the wire?
2. Are there any good materials out there that explain the protocol?  
Figuring it out by reading the source and break pointing the code is 
slow going.  Esp with the epoll threads making tracking things down hard.

Thanks for the help!
-Chris