JobTracker用户界面未显示Hadoop作业的进度 [英] JobTracker UI not showing progress of hadoop job

问题描述

我正在单节点集群下测试我的MR作业. 一旦我安装了mahout 9版本，Mapreduce作业就会停止在jobtracker中显示进度.(不知道安装mahout之后是否发生了这种情况)

I am testing my MR jobs under a single node cluster. Once I installed mahout 9 version Mapreduce jobs stopped showing the progress in jobtracker.(Dont know if that happened after mahout installation)

每当我在hadoop集群中运行作业时，它都不会像以前那样在作业跟踪程序UI中显示状态，并且控制台中显示的执行日志也不同(类似于mahout日志)

When ever I run a job in my hadoop cluster it wont show the status in job tacker UI as previous and the execution log displaying in the console is also different (similar to mahout logs)

为什么会这样?

谢谢.

推荐答案

很可能您的工作可能正在使用

Most probably you job might be running using LocalJobRunner. If your job is using LocalJobRunner you'll notice mapred.LocalJobRunner events in the mapreduce job output. This could of couple of reasons:

对于MapReduce v1 :

确保您的mapred-site.xml具有属性mapred.job.tracker，并指向相应的JobTracker主机和端口:

Make sure your mapred-site.xml has the property mapred.job.tracker and is pointing to appropriate JobTracker host and port:

<property>  
  <name>mapred.job.tracker</name>  
  <value>[fqdn_of_jobtracker]:[port_of_jobtracker]</value>  
</property>

通常，如果hadoop无法读取属性mapreduce.framework.name或将属性设置为local，则mapreduce将选择LocalJobRunner.

Generally, LocalJobRunner will be choosen by the mapreduce if the hadoop cannot read the property mapreduce.framework.name or if the property has been set to local.

如果此操作没有成功，请尝试使用hadoop的通用命令行选项明确指定属性，例如:

If this does not pan out then try explicitly specifying the property using hadoop's generic command line option like:

hadoop jar hadoop-examples.jar pi -jt [fqdn_of_jt]:[port_of_jt] 2 1000

对于YARN和MapReduce v2 :

如果在这种情况下在YARN上运行MapReduce，则如果未在mapred-site.xml中指定mapreduce框架名称，则作业将由LocalJobRunner运行，因此请确保您具有以下属性:

If you are running MapReduce on top of YARN in that case, the job would be run by LocalJobRunner if you haven't specified the mapreduce framework name in mapred-site.xml, so make sure you have the following property:

<property>  
  <name>mapreduce.framework.name</name>  
  <value>yarn</value>  
</property>

这篇关于JobTracker用户界面未显示Hadoop作业的进度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！