应用程序管理器和应用程序主文件在YARN中的区别？ [英] Difference between Application Manager and Application Master in YARN?

问题描述

我了解MRv1是如何工作的。现在我正在尝试理解MRv2。应用程序管理器和应用程序主文件在YARN中有什么不同？

解决方案应用程序主文件和应用程序管理器这两个术语经常互换使用。实际上，Application Master是请求，启动和监视应用程序特定资源的主要容器，而Application Manager是ResourceManager中的一个组件。关于Application Manager的更多细节如下。

ApplicationsManager负责维护提交的
应用程序的集合。提交应用程序后，它首先验证应用程序的
规范，并拒绝任何请求为其
ApplicationMaster提供不可满足的资源的应用程序（即集群中没有足够资源的
节点运行ApplicationMaster本身）。然后确保没有其他应用程序已经使用相同的应用程序ID提交
，这可能是由错误的
或恶意客户端导致的。最后，它将允许的应用程序转发给调度程序。
在从ResourceManager的
内存完全撤出之前，此组件还负责记录和管理已完成的应用程序
一段时间。当应用程序完成时，它将ApplicationSummary放置在守护进程的
日志文件中。
最后，在
应用程序完成后，ApplicationsManager会保存已完成应用程序的缓存，以支持用户对应用程序数据的请求（通过Web UI或命令
行）。配置属性yarn.resourcemanager.max-completed-applications
控制ResourceManager
在任何时间点记住的最大数量的已完成应用程序。缓存是先入先出的列表，最旧的应用程序
被移出以容纳新鲜完成的应用程序。

参考：Hadoop YARN Book

I understood how MRv1 works.Now I am trying to understand MRv2.. what's the difference between Application Manager and Application Master in YARN?

解决方案

The terms Application Master and Application Manager are often used interchangeably. In reality Application Master is the main container requesting, launching and monitoring application specific resources, whereas Application Manager is a component inside ResourceManager. More details about Application Manager is given below.

The ApplicationsManager is responsible for maintaining a collection of submitted applications. After application submission, it first validates the application’s specifications and rejects any application that requests unsatisfiable resources for its ApplicationMaster (i.e., there is no node in the cluster that has enough resources to run the ApplicationMaster itself). It then ensures that no other application was already submitted with the same application ID—a scenario that can be caused by an erroneous or a malicious client. Finally, it forwards the admitted application to the scheduler. This component is also responsible for recording and managing finished applications for a while before they are completely evacuated from the ResourceManager’s memory. When an application finishes, it places an ApplicationSummary in the daemon’s log file. Finally, the ApplicationsManager keeps a cache of completed applications long after applications finish to support users’ requests for application data (via web UI or command line). The configuration property yarn.resourcemanager.max-completed-applications controls the maximum number of such finished applications that the ResourceManager remembers at any point of time. The cache is a first-in, first-out list, with the oldest applications being moved out to accommodate freshly finished applications.

Reference: Hadoop YARN Book

这篇关于应用程序管理器和应用程序主文件在YARN中的区别？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！