[Gluster-devel] Language Bindings and Data Serialization in Gluster

[Brad Childs]

I'm working on a hadoop/gluster integration.  First, whoever wrote the initial map/reduce gluster DFS layer did a great job.  Same with the FUSE layer--  FUSE alone solves a lot of problems.

However, some of the hadoop architectures require cross language (java) bindings with gluster (c). Other's I've talked to about cross language bindings have shown interest in gluster--> python and ruby bindings.  From the research I've done, most of a common language interface is simple and the only tricky part is data serialization.  

I'm not an expert in data serialization across programming languages so it's been a lot of research.  There's a few new and old solutions to the problem but I think the older non IDL solutions are obvious and not really that interesting (RPC, ASN.1, raw JNI or other language specific native access). 

Moving up to IDL abstractions, there's Thrift, Protobuf and SWIG.  SWIG is really good for simple multi-language bindings but requires manual data serialization.

>From my research the common solutions for multi-language serialization today is either protobuf or thrift.   Thrift is a facebook originated solution (now under apache) and protobuf is google's open solution.

I'm leaning towards thrift since it's apache.  A few of the hadoop architectures already support thrift so that would simplify things.  What's your opinion?  Is anyone familiar with, or already solved I/O serialization across programming languages in gluster? I'd love some feedback and discussion.  

Links:
SWIG: http://www.swig.org/
Protobuf: http://code.google.com/p/protobuf/
Serialization Discussion: http://stackoverflow.com/questions/4633611/what-are-the-key-differences-between-apache-thrift-google-protocol-buffers-mes
Previous Gluster Interface Pitch: https://fedoraproject.org/wiki/Summer_coding_ideas_for_2012#Implement_a_binding_translator_for_GlusterFS

Humor: http://www.quickmeme.com/meme/3t05wa/


Thanks!
bradley childs