YARN Resourcemanager不连接到nodemanager [英] YARN Resourcemanager not connecting to nodemanager

问题描述

Hadoop 2.2
zookeeper 3.4.5
Hbase 0.96
Hive 0.12

当我访问http：//：50070时，我能够正确地看到2个节点正在运行。

问题是，当我转到http：//：8088时，它显示0个节点正在运行。

据我所知：8088反映了resourcemanager并显示了正在运行的nodemanagers的数量。守护进程都开始了，但看起来nodemanagers没有连接到资源管理器。

这是日志文件：

  2013-12-16 20:55:48,648 INFO org.apache.hadoop.yarn.client.RMProxy：连接到ResourceManager，位于/127.0.0.1:8031 
 2013-12-16 20：55：49,755 INFO org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1：8031。已经尝试0次（s）;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
 2013-12-16 20：55：50,756信息org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1： 8031。已经尝试过1次;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
 2013-12-16 20:55:51,757 INFO org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1： 8031。已经尝试过2次（s）;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
 2013-12-16 20:55:52,758信息org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1： 8031。已经尝试过3次;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
 2013-12-16 20:55:53,759信息org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1： 8031。已经尝试过4次;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
 2013-12-16 20:55:54,760 INFO org.apache.hadoop.ipc.Client：重试连接到服务器：localhost / 127.0.0.1： 8031。已经尝试过5次;重试策略是RetryUpToMaximumCountWithFixedSleep（maxRetries = 10，sleepTime = 1 SECONDS）
  

打开。

编辑：

对于未来的人来说，我需要编辑我的纱线网站.xml看起来像下面这样：

 < property> 
< name> yarn.nodemanager.aux-services< / name> 
< value> mapreduce_shuffle< /值> 
< / property> 
<属性> 
< name> yarn.nodemanager.aux-services.mapreduce.shuffle.class< / name> 
< value> org.apache.hadoop.mapred.ShuffleHandler< / value> 
< / property> 
<属性> 
< name> yarn.resourcemanager.scheduler.address< / name> 
<值> master-1：8030< /值> 
< / property> 
<属性> 
< name> yarn.resourcemanager.address< / name> 
<值> master-1：8032< /值> 
< / property> 
<属性> 
< name> yarn.resourcemanager.webapp.address< / name> 
<值> master-1：8088< /值> 
< / property> 
<属性> 
< name> yarn.resourcemanager.resource-tracker.address< / name> 
<值> master-1：8031< /值> 
< / property> 
<属性> 
< name> yarn.resourcemanager.admin.address< / name> 
<值> master-1：8033< /值> 
< / property> 
  

解决方案

您需要指定

主机名：端口

是的，必须设置在所有节点上，不仅仅是ResourceManager节点（只需将同一个hadoop的conf目录复制到所有节点即可）。如果您需要一些最低限度的工作配置，请点击此处查看： toster.ru/q/57046#answer_208326

thanks in advance for any help

I am running the following versions:

Hadoop 2.2 zookeeper 3.4.5 Hbase 0.96 Hive 0.12

When I go to http://:50070 I am able to correctly see that 2 nodes are running.

The problem is when I go to http://:8088 it shows 0 nodes running.

I understand that :8088 reflects the resourcemanager and shows the number of nodemanagers running. The daemons all start, but it would appear that the nodemanagers aren't connecting to the resourcemanager.

This is the log file:

2013-12-16 20:55:48,648 INFO org.apache.hadoop.yarn.client.RMProxy: Connecting to ResourceManager at /127.0.0.1:8031
2013-12-16 20:55:49,755 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:50,756 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:51,757 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:52,758 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:53,759 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
2013-12-16 20:55:54,760 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

I have checked and port 8031 is open.

EDIT:

For people viewing this in the future, I needed to edit my yarn-site.xml to look like the following:

<property>
   <name>yarn.nodemanager.aux-services</name>
   <value>mapreduce_shuffle</value>
</property>
<property>
   <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
   <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
   <name>yarn.resourcemanager.scheduler.address</name>
   <value>master-1:8030</value>
</property>
<property>
   <name>yarn.resourcemanager.address</name>
   <value>master-1:8032</value>
</property>
<property>
   <name>yarn.resourcemanager.webapp.address</name>
   <value>master-1:8088</value>
</property>
<property>
   <name>yarn.resourcemanager.resource-tracker.address</name>
   <value>master-1:8031</value>
</property>
<property>
   <name>yarn.resourcemanager.admin.address</name>
   <value>master-1:8033</value>
</property> 

解决方案

You'll need to specify

hostname:port

Yes, it has to be set on all the nodes, not just the ResourceManager node (You'll be fine just copying the same hadoop's conf dir to all of your nodes). If you need some minimal working configs to get started, take a look here: toster.ru/q/57046#answer_208326

这篇关于YARN Resourcemanager不连接到nodemanager的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！