[Gluster-devel] Removing dict from the RPC protocol

[Jeff Darcy]

> This is probably a controversial topic to bring up but, I think the dict
> that's in the RPC protocol needs to be removed. It generates a lot of
> confusion in the code because the dict is opaque. The XDR protocol has great
> support for serializing structs.
That's great when the set of things you're sending over the wire is well known and rarely changes, as is typically the case for standardized protocols such as NFS. Our usage doesn't fit that model. Many translators use dict values that have meaning only to that translator or a closely allied one, and modularity requires that we not expose internal details to other modules. Unfortunately, XDR forces us to do the exact opposite. To use your example, that would mean quotad's internal communication becomes part of a global protocol definition. Not only does that violate "separation of concerns" but it increases merge conflicts whenever those files are touched, it makes protocol versioning more complicated, etc. 

If anything, I'd say we need *less* XDR in our system. There are plenty of alternatives that handle modern protocol-design issues such as versioning and extensibility and optional values better. There are even alternatives at the next level up that would be more robust than what we have now with respect to IPv6, multi-homed hosts, epoll loops, retries, ping timers, and so on. They might improve performance too. XDR seemed fine when I first worked with it in 1990 or so, but it's not really a good basis for what what we're doing in 2015. 

> If the dict was replaced with a struct it would be a lot easier to work with.
> If someone was trying to figure out the Gluster RPC protocol to create a new
> library to talk to Gluster the dict becomes a large problem. Every time
> there is a dict in the protocol you have to search for where the code is
> used in the Gluster source code and figure out what parameters are actually
> being added to the dictionary.

> Here's an example:

> Getting quota information from quotad. The dict requires the following values
> that can only be found in the code by digging/breakpointing the daemon:
> gfid
> type
> volume-uuid
> version
> Without these values quotad rejects your rpc request as invalid. How is
> someone supposed to find this information if it's only generated at runtime?
The same way they do for just about every REST/HTML/JSON API which has the same behavior. That information truly should exist somewhere besides the code, but that doesn't have to mean entombing it in a formal IDL (of which XDR is only one example). A spec (for each module-private protocol) would be nice. Introspection would be even better. Neither would exacerbate the problems that would result from adding even more XDR definitions. 

> The quotad rpc call could be replaced with a call that has an explicit struct
> to fill in. That way everyone knows what will be passed and it's easy to
> encode/decode with any XDR library. Thoughts?
Despite what I've said, in many cases using an XDR-defined struct instead of dict values would be a good idea. Specifically, that would be true in the case where our usage *does* match the XDR strong points I mentioned right at the start. Quotad might well be such an example. On the other hand, there are many other cases where we're still better off using dicts (unless/until we adopt a better RPC foundation). My preference would be to work on improving specific cases, but not try to get rid of dicts entirely. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20150819/cf383fb2/attachment.html>