[Gluster-users] hadoop gluster

[Shubhendu Tripathi]

You should submit jobs as a user other than yarn which is also a member of the hadoop group. We usually add a mapred user.

Also check your <Hadoop Home>/etc/hadoop/container-executor.cfg:

yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=yarn
min.user.id=1000
allowed.system.users=mapred

You'll want the mapred UID > 1000 or else adjust the setting in the file.

Regards,
Shubhendu

On 06/08/2015 07:44 AM, 中川智之 wrote:
> Hi　EveryOne.
>
> I want to use Hadoop 2.x with GlusterFS. To testing, Prepare these
> softwares.
>
> <Cnfiguration>
> - CentOS 7.1(on VMWare)
> - Gluster FS 3.7
> - Hadoop 2.x
> - Glusterfs-Hadoop plugin 2.3.13
>
> Test Process is this.
>
> <Test Process>
> 1. Install CentOS 7.1. (3 VM Machine)
>    - 1 Machine is GlusterFS Client, and hadoop Namenode
>    - 2 Machine is GlusterFS BrickNode, and hadoop Datanode
> 2. Install GlusterFS 3.7, and cnfigure brick.
> 3. Mount Gluster volume from client Machine.
> 4. Install Hadoop 2.x, and HDFS configure.
>    * Hadoop installed by root.
> 5. Starting and testing HDFS and Mapreduce.
> 6. Configure Hadoop 2.0 with GlusterFS
>
> Finish and Succcess Test 5, and I started Test 6 with this Note.
>
> <Note>
> http://www.gluster.org/community/documentation/index.php/Hadoop
> https://forge.gluster.org/hadoop/pages/Configuration
>
> But I'm not entirely the way.
> - How to use glusterfs-hadoop-2.3.13.jar ?
> - Only put glusterfs-hadoop-2.3.13.jar and edit core-site.xml, Hadoop can
> use on GlusterFS?
> - If Finished Configuration, Hadoop can access on GlusterFS Volume by hadoop
> command ?
>   ex)
>   $ bin/hadoop dfs -cat <GlusterVolume>/data
>   $ bin/hadoop jar
> share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.0.jar grep
> <GlusteVolume>/data <GlusteVolume>/dataout '[a-z.]+'
>
> Please tll me the way of Configuring.
> I'm a hadoop Beginner, and I'm not so good at English.
>
>
>
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users at gluster.org
> http://www.gluster.org/mailman/listinfo/gluster-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.gluster.org/pipermail/gluster-users/attachments/20150609/4094704b/attachment.html>