ThreadPoolExecutor与ForkJoinPool:窃取子任务 [英] ThreadPoolExecutor vs ForkJoinPool: stealing subtasks

问题描述

从Java文档开始，

ForkJoinPool与其他类型的ExecutorService的不同之处主要在于采用了工作窃取:池中的所有线程都试图查找并执行由其他活动任务创建的子任务(如果不存在，则最终阻塞等待工作).

A ForkJoinPool differs from other kinds of ExecutorService mainly by virtue of employing work-stealing: all threads in the pool attempt to find and execute subtasks created by other active tasks (eventually blocking waiting for work if none exist).

当大多数任务产生其他子任务时(这与大多数ForkJoinTasks一样)，这可以实现高效处理.在构造函数中将asyncMode设置为true时，ForkJoinPools也可能适用于从未加入的事件样式任务.

This enables efficient processing when most tasks spawn other subtasks (as do most ForkJoinTasks). When setting asyncMode to true in constructors, ForkJoinPools may also be appropriate for use with event-style tasks that are never joined.

经过以下 ForkJoinPool示例，与ThreadPoolExecutor不同，我没有看到用于设置队列大小的参数.我对如何使用ForkJoinPool窃取机制一无所知.

After going through below ForkJoinPool example, Unlike ThreadPoolExecutor, I have not seen parameter to set Queue size. I did not get clue on how ForkJoinPool stealing mechanism.

//creating the ThreadPoolExecutor

ThreadPoolExecutor executorPool = new ThreadPoolExecutor(2, 10, 60, TimeUnit.SECONDS, 
new ArrayBlockingQueue<Runnable>(3000), threadFactory, rejectionHandler);

假定我已创建具有10个线程的ThreadPoolExecutor，并且已提交3000个可调用任务.这些线程如何分担子任务的执行负担?

对于相同的用例，ForkJoin池的行为如何?

推荐答案

在ForkJoinPool中，有两种队列:一种是在提交任务时基本使用的缓冲池，另一种是线程专用的(即用于每个线程).您可以从ForkJoinTask调用新任务(通常是问题的一部分).

In ForkJoinPool, there are two kinds of queues — the pool one which you basically used when submitting a task, and the thread specific one (i.e. one for each thread). From a ForkJoinTask you can invoke new tasks (generally a split of your problem).

这些新任务不会提供给池队列，而是提供给特定于线程的队列.因此，它们优先于池1进行拉/拉，就好像您已在同一任务中完成所有工作一样.此外，调用程序任务似乎被阻止以完成子任务.

These new tasks are not offered to the pool queue but to the thread specific one. Thus, they are taken/pulled in priority to the pool one, as if you have done all the job in the same task. Furthermore, the invoker task appears to be blocked for subtask completion.

实际上，阻塞时间"是用来消耗子任务的.当其他线程被工作淹没时，让其他线程闲逛"将是愚蠢的.因此，发生了偷工作".

In reality, the "blocked time" is spent to consume subtasks. It will be stupid to let other threads "to loaf around" while one of them is flooded by work. So, "work stealing" takes place.

超越.为了提高效率，窃取工作"从相反的边界接/拉任务.这样可以大大减少在队列写入过程中的争用.

To go beyond. To be efficient, "work stealing" takes/pulls task from the opposite bound. This greatly reduces contention over queue writing.

总在效率上，最好只将问题分成两个子任务，然后一次又一次地拆分子任务.即使您知道该问题也必须直接分成N个部分.这是因为窃取工作"需要并发写入共享资源，因此限制了它的激活和争用！

Always in efficiency, it's better to only split the problem in two subtasks and let the subtask split again and again. Even if you know the problem must be split directly in N parts. This is because "work stealing" requires concurrent writes to a shared resource, so limit its activation and contention!

这篇关于ThreadPoolExecutor与ForkJoinPool:窃取子任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！