无法在 master 上启动节点管理器 [英] Unable to start a node manager on master

问题描述

我正在设置一个 Hadoop YARN 集群，并且我正在使用一台机器作为主机和从机.当我使用以下命令启动 YARN 时，它会在从属节点上启动节点管理器，但不会在主节点上启动.

I am setting up a Hadoop YARN cluster and I am using a machine as both a master and a slave. When I start the YARN using the following command, it starts the nodemanager on slaves but not on the master node.

sbin/yarn-daemons.sh start nodemanager

我有一个master，也是slave，然后我在集群中有另外两个slave，slave中的nodemanagers正在正常启动.

I have a master which also is slave and then I have another two slaves within the cluster, the nodemanagers in the slaves are starting properly.

我得到的错误:

org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [0.0.0.0:8040] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException

部分命令的输出.

cat /etc/services | grep 8040
ampify          8040/tcp                # Ampify Messaging Protocol
ampify          8040/udp                # Ampify Messaging Protocol

lsof -i tcp:8040
COMMAND   PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
java    28021   df  195u  IPv6 3580602      0t0  TCP server1.mydomain.com:ampify (LISTEN

推荐答案

在 Hadoop 提供的默认配置下，端口 8040 是 NodeManager 用于本地化程序的端口.这基本上是一个服务器端点，负责将运行容器所需的文件带到本地节点上.(例如，这可以是 MapReduce 作业的 jar 文件或分布式缓存文件.)

Under the default configuration that Hadoop ships, port 8040 is the port that the NodeManager uses for the localizer. This is basically a server endpoint responsible for bringing the files required to run a container onto the local node. (For example, this can be a MapReduce job's jar file or distributed cache files.)

假设机器上有另一台服务器(此处显示为 Ampify)合法绑定到端口 8040，并且您不想停止该服务，那么可以重新配置 NodeManager 使用的本地化端口.在您的 yarn-site.xml 文件中设置属性 yarn.nodemanager.localizer.address.这记录在这里:

Assuming that there is another server on the machine (here shown as Ampify) legitimately bound to port 8040, and you don't want to stop that service, then it is possible to reconfigure the port used by the NodeManager for the localizer. Set property yarn.nodemanager.localizer.address in your yarn-site.xml file. This is documented here:

http://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

从 Hadoop 树中的 XML 源中提取，这里是该属性的文档:

Pulling that from the XML source in the Hadoop tree, here is the documentation for the property:

<property>
  <description>Address where the localizer IPC is.</description>
  <name>yarn.nodemanager.localizer.address</name>
  <value>${yarn.nodemanager.hostname}:8040</value>
</property>

这篇关于无法在 master 上启动节点管理器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！