当示例在hadoop 0.23.0上运行时挂起的作业 [英] Job hanging when example run on hadoop 0.23.0

问题描述

我试图在hadoop 0.23.0中添加容量调度程序，并试图运行一个样本pi，randomwriter程序。

所有的守护进程都正常运行，但工作正在绞死，没有更多的输出显示出来。

我无法看到积累的日志。任何人都可以让我知道这个工作挂起的原因，以及存储日志的位置。

  2012-06-08 18:41：06,118信息mapred.YARNRunner（YARNRunner.java:createApplicationSubmissionContext(355）） - 命令为ApplicationMaster启动容器的是：$ JAVA_HOME / bin / java -Dlog4j.configuration = container-log4j.properties -Dyarn.app.mapreduce.container.log.dir =< LOG_DIR> -Dyarn.app.mapreduce.container.log.filesize = 0 -Dhadoop.root.logger = INFO，CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>< LOG_DIR> / stdout 2><< ; LOG_DIR> / stderr 
 2012-06-08 18：41：06,251信息mapred.ResourceMgrDelegate（ResourceMgrDelegate.java:submitApplication(304）） - 提交应用程序application_1339151256291_0003到ResourceManager 
 2012-06-08 18： 41：06,355信息mapreduce.Job（Job.java:monitorAndPrintJob(1207）） - 正在运行的作业：job_1339151256291_0003 
 2012-06-08 18:41：07,366信息mapreduce.Job（Job.java:monitorAndPrintJob(1227）） -  map 0％reduce 0％
  

解决方案

在 http://www.thecloudavenue.com/search?q=0.2 上成功运行Hadoop-0.23.4在一个小的3节点集群上。

检查

$ b中的日志文件$ b

$ HADOOP_HOME / logs

文件夹，以查看主站和从站上的任何错误。

检查以下控制台

// master：50070 / dfshealth.jsp

检查是否

// master：50070 / dfsnodelist.jsp？whatNodes = LIVE

中DataNode的数量是正确的

检查NodeManagers的数量是否在 $ b

// master中正确报告：8088 / cluster

另外，NodeManagers的数量应该在
< blockquote>

// master：8088 / cluster / nodes

验证输出文件夹的内容是否正确已通过NameNode Web控制台创建
$ b

// master：50070 / nn_browsedfscontent.jsp

I am trying to add capacity scheduler in hadoop 0.23.0 and trying to run a sample pi, randomwriter program.

All the daemons are up and working fine, but the job is getting hanged and no more output is getting displayed.

I couldnt able to see the logs where they are accumulated. Can anyone please let me know the reason for this hanging of the job, and location where the logs are stored.

2012-06-08 18:41:06,118 INFO  mapred.YARNRunner (YARNRunner.java:createApplicationSubmissionContext(355)) - Command to launch container for ApplicationMaster is : $JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=<LOG_DIR> -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1536m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr 
2012-06-08 18:41:06,251 INFO  mapred.ResourceMgrDelegate (ResourceMgrDelegate.java:submitApplication(304)) - Submitted application application_1339151256291_0003 to ResourceManager
2012-06-08 18:41:06,355 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1207)) - Running job: job_1339151256291_0003
2012-06-08 18:41:07,366 INFO  mapreduce.Job (Job.java:monitorAndPrintJob(1227)) -  map 0% reduce 0%

解决方案

I followed the instructions on http://www.thecloudavenue.com/search?q=0.23 and successfully run Hadoop-0.23.4 on a small 3 node cluster.

Check the log files in the

$HADOOP_HOME/logs

folder for any errors on the master and the slaves.

Check for the following console

//master:50070/dfshealth.jsp

Check if the number of DataNodes is correct in

//master:50070/dfsnodelist.jsp?whatNodes=LIVE

Check if the number of NodeManagers is correctly reported in

//master:8088/cluster

Also the number of NodeManagers should be correctly specified in

//master:8088/cluster/nodes

Verify that the output folder with the proper contents has been created through the NameNode Web console

//master:50070/nn_browsedfscontent.jsp

这篇关于当示例在hadoop 0.23.0上运行时挂起的作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！