作业继续在Cloudera 5.1下的LocalJobRunner中运行 [英] Job keeps running in LocalJobRunner under Cloudera 5.1

问题描述

需要一些快速帮助。我们的作业在MapR下运行良好，但是当我们在Cloudera 5.1上启动相同的作业时，它仍然以本地模式运行。

我确定这是某种配置问题。

  14/08/22 12:16:58信息mapreduce.Job：map 0％reduce 0 ％
 14/08/22 12:17:03信息mapred.LocalJobRunner：map> map 
 14/08/22 12:17:06信息mapred.LocalJobRunner：map> map 
 14/08/22 12:17:09信息mapred.LocalJobRunner：map> map 
  

谢谢。

解决方案

问题是Cloudera 5.1运行'Yarn'分发&如果你想启动一个MapReduce作业，你不应该使用：

java -cp $ CLASSPATH CLASSNAME（在Yarn下这将开始作业）

相反，您必须使用

纱线 -cp $ CLASSPATH CLASSNAME

另外，我学到的另一件事是，您可以通过运行以下命令来获取HADOOP类路径：

纱线类路径

Need some quick help. Our job runs fine under MapR, but when we start the same job on Cloudera 5.1, it keeps running in Local mode.

I am sure this is some kind of configuration issue. Which config setting is it?

14/08/22 12:16:58 INFO mapreduce.Job: map 0% reduce 0%
14/08/22 12:17:03 INFO mapred.LocalJobRunner: map > map
14/08/22 12:17:06 INFO mapred.LocalJobRunner: map > map
14/08/22 12:17:09 INFO mapred.LocalJobRunner: map > map

Thanks.

解决方案

Problem was that Cloudera 5.1 runs 'Yarn' distribution & under Yarn if you want to start a MapReduce job you SHOULD NOT use:

java -cp $CLASSPATH CLASSNAME (Under Yarn this will start job in the Local mode.)

Instead you've to use

yarn -cp $CLASSPATH CLASSNAME

Also, another thing I learned is that, you can get the HADOOP classpath by running the following command:

yarn classpath

这篇关于作业继续在Cloudera 5.1下的LocalJobRunner中运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！