Hadoop 2.0 ApplicationMaster与NodeManager [英] Hadoop 2.0 ApplicationMaster vs NodeManager

问题描述

我无法确定Hadoop 2.0架构中 ApplicationMaster 和 NodeManager 之间的区别.

我知道ApplicationMaster负责运行 map 和 reduce 任务，并且它通过与ResourceManager协调来检索容器以运行这些任务.

但是我对NodeManager的目的感到困惑.NodeManager是为ApplicationMaster创建容器(运行这些映射和减少任务)还是ResourceManager为ApplicationMaster创建容器(运行那些映射和减少任务)，而NodeManager只是安全地管理节点的运行状况容器用于ApplicationMaster的文件和目录，并监视容器的资源使用情况等.

解决方案

ResourceManager(每个集群一个)-在所有应用程序之间仲裁资源.

NodeManager-负责容器的启动，监视其资源使用情况(cpu，内存，磁盘，网络)，并将其报告给ResourceManager/Scheduler.

因此，在回答您的问题时，ResourceManager不会自行启动容器，而是会找到合适的NodeManager来执行此操作.

下面有一些图片和更多详细信息的链接.

I am having trouble identifying the differences between the ApplicationMaster and NodeManager in the Hadoop 2.0 architecture.

I know that the ApplicationMaster is responsible for running the map and reduce tasks and it retrieves containers to run these tasks by coordinating with the ResourceManager.

But I am confused on the purpose of the NodeManager. Does the NodeManager create the containers for the ApplicationMaster (to run those map and reduce tasks) or does the ResourceManager create the containers for the ApplicationMaster (to run those map and reduce tasks) and the NodeManager just manages the health of the node, securely place files and directories needed by containers for the ApplicationMaster, and monitor the container's resource usage, etc.

解决方案

ResourceManager(one per cluster) - arbitrates resources among all the applications.

NodeManager - responsible for containers launch, monitoring their resource usage (cpu, memory, disk, network) and reporting it to the ResourceManager/Scheduler.

So, in answer to your question, ResourceManager doesn't launch containers itself it finds the suitable NodeManager that can do that.

Some pictures and links with more details bellow.

YARN Architecture from official Hadoop docs:

Anatomy of a YARN Application Run(from "Hadoop: The Definitive Guide" by Tom White):

这篇关于Hadoop 2.0 ApplicationMaster与NodeManager的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！