[Gluster-users] C/C++ lib equivalent of hadoop-glusterfs

[Jeff Darcy]

On Sat, 2012-06-16 at 15:38 -0400, Vlad wrote:
> Greetings,
> 
> I am not sure I understand fully the relationship b/w glusterfs
> project per se and https://github.com/gluster/hadoop-glusterfs, but
> I'd like to follow up on the "Hadoop Connector" mention here
> (http://www.gluster.org/community/documentation/index.php/Hadoop) and
> getFileBlockLocations API mention here
> (http://community.gluster.org/q/how-gluster-supports-map-reduce-in-absence-of-metadata-of-input-data-chunks-spread-over-multiple-machines-see-desciption/):
> 
> - does 3.3 install any kind of a C lib that would allow for "where
> data actually landed" queries?
> 
> This would be very useful to users who want to structure their HPC
> jobs in the map/reduce style, but without using Hadoop specifically.
> (Really, the main innovation in MapReduce is colocation of calculation
> and data and I'd rather use a fs that's mountable in the classic sense
> as opposed to HDFS).

The getFileLocations API is implemented using a "magic"
extended-attribute request.  The extended attribute is
trusted.glusterfs.pathinfo; if you try to fetch that, we dynamically
construct a reply describing where the data went.  I suppose we could
provide a C/C++ library to parse that into some sort of structure that's
easier to use from within a program, but AFAIK that has not been done.