如何用hadoop-2.2在oozie中执行mapreduce程序 [英] How execute a mapreduce programs in oozie with hadoop-2.2

问题描述

在ubuntu中2.2.0和oozie-4.0.0。我无法在oozie中执行mapreduce程序。

我是oozie中jobtracker 8032的uisng资源管理器端口号。

在oozie中调度运行状态和运行在纱线也有一段时间后，我得到这样的错误（下面）在hadoop日志，仍然在oozie日志中运行

错误：

  2014-05-30 10：38：14,322 INFO [RMCommunator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor： getResources（）for application_1401425739447_0003：ask = 3 release = 0 newContainers = 0 finishedContainers = 0 resourcelimit =< memory：1024，vCores：-1> knownNMs = 1 
 2014-05-30 10：38：17,296 INFO [Socket读写器＃1用于端口47412] SecurityLogger.org.apache.hadoop.ipc.Server：Auth成功job_1401425739447_0003（auth：SIMPLE）
二零一四年五月三十〇日10：38：17316 INFO [上47412 IPC服务器处理器0] org.apache.hadoop.mapred.TaskAttemptListenerImpl：JVM与ID：jvm_1401425739447_0003_m_000002问了任务
二零一四年五月三十〇日10 ：38：17316 INFO [上47412 IPC服务器处理程序0] org.apache.hadoop.mapred.TaskAttemptListenerImpl：JVM与ID：jvm_1401425739447_0003_m_000002给定的任务：attempt_1401425739447_0003_m_000000_0 
 2014年5月30日10：38：22524 INFO [IPC服务器上47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl处理机1：平从attempt_1401425739447_0003_m_000000_0 
 2014年5月30日10：38：25996 INFO上47412 [IPC服务器处理程序2] org.apache.hadoop.mapred.TaskAttemptListenerImpl ：从try_1401425739447_0003_m_000000_0 
状态更新2014-05-30 10：38：26,003信息[47412上的IPC服务器处理程序2] org.apache.hadoop.mapred.T askAttemptListenerImpl：TaskAttempt attempt_1401425739447_0003_m_000000_0的进步是：1.0 
二零一四年五月三十零日10：38：29066 INFO [IPC服务器上47412处理机3] org.apache.hadoop.mapred.TaskAttemptListenerImpl：平从attempt_1401425739447_0003_m_000000_0 
 2014 -05-30 10：38：32071 INFO [上47412 IPC服务器处理程序4] org.apache.hadoop.mapred.TaskAttemptListenerImpl：平从attempt_1401425739447_0003_m_000000_0 
 2014年5月30日10：38：35075 INFO [IPC服务器处理程序5上47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl：平从attempt_1401425739447_0003_m_000000_0 
 2014年5月30日10：38：38079 INFO [IPC服务器上47412处理机6] org.apache.hadoop.mapred.TaskAttemptListenerImpl： Ping from attempt_1401425739447_0003_m_000000_0 
此错误继续... 
  

我能够运行java示例如果我试图运行猪，蜂巢，sqoop或我自己的java mapreduce程序，我得到了abo错误我不知道它为什么来。我已经在oozie中提供了我的hadoop配置路径，并开始了我的jobhistory服务器。

请帮助我...

运行在同一系统中的NodeManager和Resource Manager。所以我得到这个错误。我在第二个系统中启动了Node manager，并解决了我的问题。

2.2.0 and oozie-4.0.0 in ubuntu. I am cant able to execute mapreduce programs in oozie.

i am uisng resource manager port number for jobtracker 8032 in oozie.

while scheduling in oozie to goes to running state and running in yarn also after some time i am getting error like this(below) in hadoop logs and still running in oozie logs

Error:

2014-05-30 10:38:14,322 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() for application_1401425739447_0003: ask=3 release= 0 newContainers=0 finishedContainers=0 resourcelimit=<memory:1024, vCores:-1> knownNMs=1
2014-05-30 10:38:17,296 INFO [Socket Reader #1 for port 47412] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1401425739447_0003 (auth:SIMPLE)
2014-05-30 10:38:17,316 INFO [IPC Server handler 0 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1401425739447_0003_m_000002 asked for a task
2014-05-30 10:38:17,316 INFO [IPC Server handler 0 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1401425739447_0003_m_000002 given task: attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:22,524 INFO [IPC Server handler 1 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:25,996 INFO [IPC Server handler 2 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Status update from attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:26,003 INFO [IPC Server handler 2 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1401425739447_0003_m_000000_0 is : 1.0
2014-05-30 10:38:29,066 INFO [IPC Server handler 3 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:32,071 INFO [IPC Server handler 4 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:35,075 INFO [IPC Server handler 5 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0
2014-05-30 10:38:38,079 INFO [IPC Server handler 6 on 47412] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1401425739447_0003_m_000000_0
this error continues...

i can able to run java example(give in oozie) mapreduce program in oozie.

if i am trying to run the pig,hive,sqoop or my own java mapreduce program i am getting the above error i dont know why it comes. i already given my hadoop configuration path in oozie and i started my jobhistory server.

Help me please...

解决方案

NodeManager and Resource Manager running in same system. So i am gettting this error. I started Node manager in second system and my problem is solved.

这篇关于如何用hadoop-2.2在oozie中执行mapreduce程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！