Yarn MapReduce作业问题 - Hadoop 2.3.0中的AM Container启动错误 [英] Yarn MapReduce Job Issue - AM Container launch error in Hadoop 2.3.0

问题描述

我已经设置了Hadoop 2.3.0的2节点集群。它的工作正常，我可以成功运行distributedshell-2.2.0.jar示例。但是，当我尝试运行任何mapreduce作业时出现错误。根据（ http：//www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide\">我已经设置了MapRed.xml和其他配置来运行MapReduce作业。 //www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide ），但我得到以下错误：

  14/03/22 20:31:17信息mapreduce.Job：作业job_1395502230567_0001由于以下原因而失败，状态为失败：应用程序application_1395502230567_0001因AM容器for appattempt_1395502230567_0001_000002退出
而失败2次exitCode：1由于：从容器启动异常：org.apache.hadoop.util.Shell $ ExitCodeException：
 org.apache.hadoop.util.Shell $ ExitCodeException：$ b $在org.apache.hadoop .util.Shell.runCommand（Shell.java:505）
位于org.apache.hadoop.util.Shell.run（Shell.java:418）
位于org.apache.hadoop.util.Shell $ ShellCommandExecutor.execute（Shell.java:650）
 at org.apache.hadoop.yarn.s erver.nodemanager.DefaultContainerExecutor.launchContainer（DefaultContainerExecutor.java:195）
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call（ContainerLaunch.java:283）
 at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call（ContainerLaunch.java:79）$ b $ at java.util.concurrent.FutureTask.run（FutureTask.java:262）
在java.util.concurrent.ThreadPoolExecutor.runWorker（ThreadPoolExecutor.java:1145）
在java.util.concurrent.ThreadPoolExecutor $ Worker.run（ThreadPoolExecutor.java:615）$ b $在java.lang。 Thread.run（Thread.java:744）
 
 
使用非零退出代码退出的容器1 
。尝试此尝试..失败应用程序。 
 14/03/22 20:31:17信息mapreduce.Job：Counters：0 
工作结束：Sat Mar 22 20:31:17 PKT 2014 
工作耗时6秒。 
  

如果看stderr（工作日志），只有一行
无法找到或加载主类614

现在我已经使用Google当你有不同的JAVA版本或 yarn-site.xml classpath没有正确设置，我的 yarn-site.xml 有这个

 < property> 
< name> yarn.application.classpath< / name> 
< value> /opt/yarn/hadoop-2.3.0/etc/hadoop,/opt/yarn/hadoop-2.3.0 / *，/ opt / yarn / hadoop-2.3.0 / lib / * ，/选择/纱线/ Hadoop的2.3.0 / *，/选择/纱线/ Hadoop的2.3.0 / lib中/ *，/选择/纱线/ Hadoop的2.3.0 / *，/选择/纱线/ Hadoop的2.3 0.0 / lib中/ *，/选择/纱线/ Hadoop的2.3.0 / *，/选择/纱线/ Hadoop的2.3.0 / lib中/ * LT; /值GT; 
< / property> 
  

那么其他想法在这里可能是什么问题？

我正在执行像这样的mapreduce作业：

  $ HADOOP_PREFIX / bin / hadoop jar $ HADOOP_PREFIX / share / hadoop / mapreduce / hadoop-mapreduce-examples-2.2.0.jar randomwriter out 
  

<解决方案我试图手动安装Hortonworks HDP 2.1时遇到了同样的问题。我设法捕获了包含以下内容的容器启动脚本：

 ＃！/ bin / bash 
 
 export NM_HTTP_PORT =8042
 export LOCAL_DIRS =/ data / 1 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001，/ data / 2 / hadoop / yarn / local / usercache / doug / appcache / data / 3 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001，/ data / 4 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001
 export JAVA_HOME =/ usr / java / latest
 export NM_AUX_SERVICE_mapreduce_shuffle =AAA0 + gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA =
 export CLASSPATH =$ PWD：$ HADOOP_CONF_DIR：$ HADOOP_COMMON_HOME / share / hadoop / common / *：$ HADOOP_COMMON_HOME / share / hadoop / common / lib目录/ *：$ HADOOP_HDFS_HOME /股/的Hadoop / HDFS / *：$ HADOOP_HDFS_HOME /股/的Hadoop / HDFS / lib目录/ *：$ HADOOP_YARN_HOME /股/的Hadoop /纱/ *：$ HADOOP_YARN_HOME /股/的Hadoop /纱/ lib目录/ *：$ HADOOP_MAPRED_HOME /股/的Hadoop / MapReduce的/ *：$ HADOOP_MAPRED_HOME /秒hare / hadoop / mapreduce / lib / *：job.jar / job.jar：job.jar / classes /：job.jar / lib / *：$ PWD / *
 export HADOOP_TOKEN_FILE_LOCATION =/ data / 2 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001 / container_1406927878786_0001_01_000001 / container_tokens
 export NM_HOST =test02.admin.hypertable.com
 export APPLICATION_WEB_PROXY_BASE =/ proxy / application_1406927878786_0001
 export JVM_PID =$$
 export USER =doug
 export HADOOP_HDFS_HOME =/ usr / lib / hadoop-hdfs
 export PWD =/ data / 2 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001 / container_1406927878786_0001_01_000001
 export CONTAINER_ID =container_1406927878786_0001_01_000001
 export HOME =/ home /
 export NM_PORT =62404
 export LOGNAME =doug
 export APP_SUBMIT_TIME_ENV =1406928095871
 export MAX_APP_ATTEMPTS =2
 export HADOOP_CONF_DIR =/ etc / hadoop / conf
 export MALLOC_ARENA_MAX = 4
 export LOG_DIRS =/ data / 1 / hadoop / yarn / logs / application_1406 927878786_0001 / container_1406927878786_0001_01_000001，/数据/ 2 /的hadoop /纱线/日志/ application_1406927878786_0001 / container_1406927878786_0001_01_000001，/数据/ 3 / hadoop的/纱线/日志/ application_1406927878786_0001 / container_1406927878786_0001_01_000001，/数据/ 4 / hadoop的/纱线/日志/ application_1406927878786_0001 / container_1406927878786_0001_01_000001\" 
 ln -sf/data/1/hadoop/yarn/local/usercache/doug/filecache/10/libthrift-0.9.2.jarlibthrift-0.9.2.jar
 ln -sf /data/4/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/filecache/13/job.xmljob.xml
 mkdir -p jobSubmitDir 
 ln -sf/ data / 3 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001 / filecache / 12 / job.splitjobSubmitDir / job.split
 mkdir -p jobSubmitDir 
 ln -sf/ data / 2 / hadoop / yarn / local / usercache / doug / appcache / application_1406927878786_0001 / filecache / 11 / job.splitmetainfojobSubmitDir / job.splitmetainfo
 ln -sf/ data / 1 / hadoop / yarn /本地/ usercache /豆豆/应用程序缓存/ APPLI cation_1406927878786_0001 / filecache / 10 / job.jarjob.jar
 ln -sf/data/2/hadoop/yarn/local/usercache/doug/filecache/11/hypertable-0.9.8.0-apache2。 jarhypertable-0.9.8.0-apache2.jar
 exec / bin / bash -c$ JAVA_HOME / bin / java -Dlog4j.configuration = container-log4j.properties -Dyarn.app.container.log。 dir = / data / 4 / hadoop / yarn / logs / application_1406927878786_0001 / container_1406927878786_0001_01_000001 -Dyarn.app.container.log.filesize = 0 -Dhadoop.root.logger = INFO，CLA -Xmx1024m org.apache.hadoop.mapreduce.v2。 app.MRAppMaster 1> / data / 4 / hadoop / yarn / logs / application_1406927878786_0001 / container_1406927878786_0001_01_000001 / stdout 2> / data / 4 / hadoop / yarn / logs / application_1406927878786_0001 / container_1406927878786_0001_01_000001 / stderr
  

设置 CLASSPATH 的行是罪魁祸首。为了解决这个问题，我必须设置变量 HADOOP_COMMON_HOME ， HADOOP_HDFS_HOME ， HADOOP_YARN_HOME 和 HADOOP_MAPRED_HOME 在 hadoop-env.sh 中指向<$ c $下的相应目录C> / usr / lib中。在每个目录中，我还必须设置共享/ hadoop /...子目录层次结构，其中可以找到罐子。

I have setup a 2 node cluster of Hadoop 2.3.0. Its working fine and I can successfully run distributedshell-2.2.0.jar example. But when I try to run any mapreduce job I get error. I have setup MapRed.xml and other configs for running MapReduce job according to (http://www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide) but I am getting following error :

14/03/22 20:31:17 INFO mapreduce.Job: Job job_1395502230567_0001 failed with state FAILED due to: Application application_1395502230567_0001 failed 2 times due to AM Container for appattempt_1395502230567_0001_000002 exited 
with  exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: 
    org.apache.hadoop.util.Shell$ExitCodeException: 
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
        at org.apache.hadoop.util.Shell.run(Shell.java:418)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)


    Container exited with a non-zero exit code 1
    .Failing this attempt.. Failing the application.
    14/03/22 20:31:17 INFO mapreduce.Job: Counters: 0
    Job ended: Sat Mar 22 20:31:17 PKT 2014
    The job took 6 seconds.

And if look at stderr (log of job) there is only one line "Could not find or load main class 614"

Now I have googled it and usually this issues comes when you have different JAVA versions or in yarn-site.xml classpath is not properly set , my yarn-site.xml has this

  <property>
    <name>yarn.application.classpath</name>
    <value>/opt/yarn/hadoop-2.3.0/etc/hadoop,/opt/yarn/hadoop-2.3.0/*,/opt/yarn/hadoop-2.3.0/lib/*,/opt/yarn/hadoop-2.3.0/*,/opt/yarn/hadoop-2.3.0/lib/*,/opt/yarn/hadoop-2.3.0/*,/opt/yarn/hadoop-2.3.0/lib/*,/opt/yarn/hadoop-2.3.0/*,/opt/yarn/hadoop-2.3.0/lib/*</value>
  </property>

So any other ideas what could be the issue here ?

I am running my mapreduce job like this:

$HADOOP_PREFIX/bin/hadoop jar $HADOOP_PREFIX/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar randomwriter out

解决方案

I encountered the same problem when trying to install Hortonworks HDP 2.1 manually. I managed to capture the container launcher script which contained the following:

#!/bin/bash

export NM_HTTP_PORT="8042"
export LOCAL_DIRS="/data/1/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001,/data/2/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001,/data/3/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001,/data/4/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001"
export JAVA_HOME="/usr/java/latest"
export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
export CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*"
export HADOOP_TOKEN_FILE_LOCATION="/data/2/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/container_1406927878786_0001_01_000001/container_tokens"
export NM_HOST="test02.admin.hypertable.com"
export APPLICATION_WEB_PROXY_BASE="/proxy/application_1406927878786_0001"
export JVM_PID="$$"
export USER="doug"
export HADOOP_HDFS_HOME="/usr/lib/hadoop-hdfs"
export PWD="/data/2/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/container_1406927878786_0001_01_000001"
export CONTAINER_ID="container_1406927878786_0001_01_000001"
export HOME="/home/"
export NM_PORT="62404"
export LOGNAME="doug"
export APP_SUBMIT_TIME_ENV="1406928095871"
export MAX_APP_ATTEMPTS="2"
export HADOOP_CONF_DIR="/etc/hadoop/conf"
export MALLOC_ARENA_MAX="4"
export LOG_DIRS="/data/1/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001,/data/2/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001,/data/3/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001,/data/4/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001"
ln -sf "/data/1/hadoop/yarn/local/usercache/doug/filecache/10/libthrift-0.9.2.jar" "libthrift-0.9.2.jar"
ln -sf "/data/4/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/filecache/13/job.xml" "job.xml"
mkdir -p jobSubmitDir
ln -sf "/data/3/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/filecache/12/job.split" "jobSubmitDir/job.split"
mkdir -p jobSubmitDir
ln -sf "/data/2/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/filecache/11/job.splitmetainfo" "jobSubmitDir/job.splitmetainfo"
ln -sf "/data/1/hadoop/yarn/local/usercache/doug/appcache/application_1406927878786_0001/filecache/10/job.jar" "job.jar"
ln -sf "/data/2/hadoop/yarn/local/usercache/doug/filecache/11/hypertable-0.9.8.0-apache2.jar" "hypertable-0.9.8.0-apache2.jar"
exec /bin/bash -c "$JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/data/4/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA  -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/data/4/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001/stdout 2>/data/4/hadoop/yarn/logs/application_1406927878786_0001/container_1406927878786_0001_01_000001/stderr "

The line that sets CLASSPATH was the culprit. To resolve the problem I had to set the variables HADOOP_COMMON_HOME, HADOOP_HDFS_HOME, HADOOP_YARN_HOME, and HADOOP_MAPRED_HOME in hadoop-env.sh to point to the appropriate directories under /usr/lib. In each of those directories I also had to setup the share/hadoop/... subdirectory hierarchy where the jars could be found.

这篇关于Yarn MapReduce作业问题 - Hadoop 2.3.0中的AM Container启动错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！