[Gluster-users] Gluster Hadoop Plugin

[Venky Shankar]

Hi,
11/03/2011 04:09 PM, Neo wrote:
> I am trying out the Hadoop Plugin with Gluster. I am trying to setup a
> debug evironment for Hadoop using Eclipse with which I will be able to
> debug Hadoop and Plugin Code.
>
> Can you give me some pointers regarding configuring the Plugin code in Hadoop?
>
> Is the plugin always compiled seperately? Or Can I add it to Hadoop code?

As of now the plugin is not a part of Hadoop codebase, you would need to 
compile it or use the RPM.

The RPM package contains the pre-compiled jar (plugin) and a sample 
configuration file. The RPM package is available here: 
http://download.gluster.com/pub/gluster/glusterfs/qa-releases/3.3-beta-2/glusterfs-hadoop-0.20.2-0.1.x86_64.rpm

For configuring the plugin see the CONFIGURATION section in this README: 
https://github.com/gluster/hadoop-glusterfs/blob/master/glusterfs-hadoop/README
> I am using the plugin code from: https://github.com/gluster/hadoop-glusterfs
yes, that's the plugin repo.
> Can you briefly explain how does the plugin code interface with Hadoop?
In short, once you have placed the plugin in Hadoop lib directory and 
configured it (conf/core-site.xml), starting the Map/Reduce daemons will 
result in Hadoop loading the plugin and using the FileSystem interface 
it provides.GlusterFS Hadoop plugin does a FUSE mount when it is 
initialized and does file I/O on it.

Let me know if you need more details.
> Thanks,
> Mak
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
Thanks,
-Venky