Flink:在Jobmanager UI中设置的并行度与任务槽有什么关系? [英] Flink: how does the parallelism set in the Jobmanager UI relate to task slots?

问题描述

比方说，我有8个任务管理器和16个任务槽.如果我使用Jobmanager UI提交作业并将并行度设置为8，是否仅使用8个任务槽?

Let's say I have 8 task managers with 16 task slots. If I submit a job using the Jobmanager UI and set the parallelism to 8, do I only utilise 8 task slots?

如果我有8个任务管理器和8个插槽，并以8的并行度提交同一作业，该怎么办?完全一样吗?还是处理数据的方式有所不同?

What if I have 8 task managers with 8 slots, and submit the same job with a parallelism of 8? Is it exactly the same thing? Or is there a difference in the way the data is processed?

谢谢.

推荐答案

Flink群集中任务插槽的总数定义了最大并行度，但是使用的插槽数可能会超过实际的并行度.例如，考虑一下这项工作:

The total number of task slots in a Flink cluster defines the maximum parallelism, but the number of slots used may exceed the actual parallelism. Consider, for example, this job:

如果在具有2个任务管理器的群集中并行运行两个，每个任务管理器提供3个插槽，则调度程序将使用5个任务插槽，如下所示:

If run with parallelism of two in a cluster with 2 task managers, each offering 3 slots, the scheduler will use 5 task slots, like this:

但是，如果基本并行度增加到6，则调度程序将执行此操作(请注意，在本示例中，接收器的并行度保持为1):

However, if the base parallelism is increased to six, then the scheduler will do this (note that the sink remains at a parallelism of one in this example):

请参见 Flink的分布式运行时环境有关更多信息.

这篇关于Flink:在Jobmanager UI中设置的并行度与任务槽有什么关系?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！