数据节点和节点管理器未以伪集群模式启动（Apache Hadoop） [英] Data Node and Node Manager is not starting in pseudo-cluster mode(Apache Hadoop)

问题描述

数据节点和节点管理器未以伪集群模式（Apache Hadoop）启动。

在日志文件中查看此错误：

  *** 2017-08-22 17：15：08,403错误org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl：意外错误开始NodeStatusUpdater 
 org.apache.hadoop.yarn.exceptions.YarnRuntimeException：从Resourcemanager收到SHUTDOWN信号，NodeManager注册失败，来自ResourceManager的消息：来自archit的NodeManager不满足最小分配，向NodeManager发送SHUTDOWN信号。 ** 
 
位于org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM（NodeStatusUpdaterImpl.java:278）
位于org.apache.hadoop.yarn.server.nodemanager。 NodeStatusUpdaterImpl.serviceStart（NodeStatusUpdaterImpl.java:197）
 at org.apache.hadoop.service.AbstractService.start（AbstractService.java:193）
 at org.apache.hadoop.service.CompositeService.serviceStart （CompositeService.java:120）
 at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart（NodeManager.java:272）
 at org.apache.hadoop.service.AbstractService.start （AbstractService.java:193）
 at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager（NodeManager.java:496）
 at org.apache.hadoop.yarn.server.nodemanager .NodeManager.main（NodeManager.java:543）
 2017-08-22 17：15：08,404 INFO org.apache.hadoop.service.AbstractService：Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl在STARTED状态下失败;原因：org.apache.hadoop.yarn.exceptions.YarnRuntimeException：
  

解决方案

来自archit的 ResourceManager：NodeManager不满足最小分配当启动节点管理器的节点没有足够的资源时，会看到错误wrt yarn.scheduler.minimum-allocation-vcores 和 yarn.scheduler.minimum-allocation-mb 配置。

减少 yarn.scheduler.minimum-allocation-vcores 和 yarn.scheduler的值.minimum-allocation-mb 然后重新启动纱线。

The Data Node and Node Manager is not starting in pseudo-cluster mode (Apache Hadoop).

Seeing this error in the log file:

***2017-08-22 17:15:08,403 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Recieved SHUTDOWN signal from Resourcemanager ,Registration of NodeManager failed, Message from ResourceManager: NodeManager from  archit doesn't satisfy minimum allocations, Sending SHUTDOWN signal to the NodeManager.***

    at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:278)
    at org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.serviceStart(NodeStatusUpdaterImpl.java:197)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:120)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStart(NodeManager.java:272)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.initAndStartNodeManager(NodeManager.java:496)
    at org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:543)
2017-08-22 17:15:08,404 INFO org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl failed in state STARTED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:

解决方案

The "ResourceManager: NodeManager from archit doesn't satisfy minimum allocations" error is seen when node on which node manager is being started does not have enough resources w.r.t yarn.scheduler.minimum-allocation-vcores and yarn.scheduler.minimum-allocation-mb configurations.

Reduce values of yarn.scheduler.minimum-allocation-vcores and yarn.scheduler.minimum-allocation-mb then restart yarn.

这篇关于数据节点和节点管理器未以伪集群模式启动（Apache Hadoop）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！