公平调度程序和容量调度程序有什么区别? [英] What is the difference between the fair and capacity schedulers?

问题描述

我是Hadoop领域的新手，想知道公平调度程序和容量调度程序之间的区别.另外，我们什么时候应该使用每个?请以简单的方式回答，因为我在Internet上阅读了很多东西，但从中学到的东西却很少.

I am new to the world of Hadoop and want to know the difference between fair and capacity schedulers. Also when are we supposed to use each one? Please answer in a simple way because I read many things on the Internet but I don't get much from them.

推荐答案

公平调度是一种为作业分配资源的方法，以使所有作业在一段时间内平均获得相等的资源份额.当有一个作业正在运行时，该作业将使用整个群集.当提交其他作业时，将腾出的任务插槽分配给新作业，以便每个作业获得大致相同的CPU时间.与默认的Hadoop调度程序形成一个工作队列不同，它可以让短时间的工作在合理的时间内完成，而不会使长时间的工作变得饥饿.这也是在多个用户之间共享集群的一种合理方法.最后，公平共享也可以与工作优先级一起使用-优先级用作权重，以确定每个工作应获得的总计算时间的比例.

Fair scheduling is a method of assigning resources to jobs such that all jobs get, on average, an equal share of resources over time. When there is a single job running, that job uses the entire cluster. When other jobs are submitted, tasks slots that free up are assigned to the new jobs, so that each job gets roughly the same amount of CPU time. Unlike the default Hadoop scheduler, which forms a queue of jobs, this lets short jobs finish in reasonable time while not starving long jobs. It is also a reasonable way to share a cluster between a number of users. Finally, fair sharing can also work with job priorities - the priorities are used as weights to determine the fraction of total compute time that each job should get.

CapacityScheduler旨在允许共享大型群集，同时为每个组织提供最小的容量保证.中心思想是，Hadoop Map-Reduce集群中的可用资源在多个组织之间进行划分，这些组织根据计算需求共同为集群提供资金.组织还有一个额外的好处，即组织可以访问任何多余的容量而不会被他人使用.这样可以以经济高效的方式为组织提供弹性.

The CapacityScheduler is designed to allow sharing a large cluster while giving each organization a minimum capacity guarantee. The central idea is that the available resources in the Hadoop Map-Reduce cluster are partitioned among multiple organizations who collectively fund the cluster based on computing needs. There is an added benefit that an organization can access any excess capacity no being used by others. This provides elasticity for the organizations in a cost-effective manner.

这篇关于公平调度程序和容量调度程序有什么区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！