Java ForkJoinPool具有非递归任务，是否可以正常工作？ [英] Java ForkJoinPool with non-recursive tasks, does work-stealing work?

问题描述

我想通过以下方法将 Runnable 任务提交到ForkJoinPool：

I want to submit Runnable tasks into ForkJoinPool via a method:

forkJoinPool.submit(Runnable task)

注意，我使用的是JDK 7.

Note, I use JDK 7.

在引擎盖下，它们被转换为ForkJoinTask对象。
我知道ForkJoinPool在递归分割成较小的任务时效率很高。

Under the hood, they are transformed into ForkJoinTask objects. I know that ForkJoinPool is efficient when a task is split into smaller ones recursively.

问题：

如果没有递归，工作窃取是否仍然在ForkJoinPool中工作？

Does work-stealing still work in the ForkJoinPool if there is no recursion?

在这种情况下是否值得？

Is it worth it in this case?

更新1：
任务很小，可能不平衡。即使对于严格相同的任务，诸如上下文切换，线程调度，停车，页面未命中等事情也会妨碍导致不平衡。

更新2：
Doug Lea在并发JSR-166兴趣组，给出一个提示：

Update 2: Doug Lea wrote in the Concurrency JSR-166 Interest group, by giving a hint on this:

这也大大改善了吞吐量当所有任务都是异步的并且
提交到池而不是分叉时，这将成为一种合理的
方式来构造actor框架，以及许多普通服务，否则你可以使用ThreadPoolExecutor来获取
。

This also greatly improves throughput when all tasks are async and submitted to the pool rather than forked, which becomes a reasonable way to structure actor frameworks, as well as many plain services that you might otherwise use ThreadPoolExecutor for.

我认为，当涉及相当小的CPU绑定任务时，由于这种优化，ForkJoinPool是可行的方法。重点是这些任务已经很小，不需要递归分解。 工作窃取有效，无论是大型还是小型任务 - 任务都可以被来自Deque尾巴忙碌的工作人员的其他免费工作人员抓住。

I presume, when it comes to reasonably small CPU-bound tasks, ForkJoinPool is the way to go, thanks to this optimization. The main point is that these tasks are already small and needn't a recursive decomposition. Work-stealing works, regardless whether it is a big or small task - tasks can be grabbed by another free worker from the Deque's tail of a busy worker.

更新3：
ForkJoinPool的可扩展性 - Akka乒乓球队的基准测试显示了很好的结果。

Update 3: Scalability of ForkJoinPool - benchmarking by Akka team of ping-pong shows great results.

尽管如此，要更有效地应用ForkJoinPool还需要进行性能调整。

Despite this, to apply ForkJoinPool more efficiently requires performance tuning.

推荐答案

ForkJoinPool 源代码有一个很好的部分叫做实现概述，阅读为了一个终极真理。以下解释是我对JDK 8u40的理解。

ForkJoinPool source code has a nice section called "Implementation Overview", read up for an ultimate truth. The explanation below is my understanding for JDK 8u40.

从第一天起， ForkJoinPool 每个工作线程都有一个工作队列（我们称之为工人队列）。分叉的任务被推送到本地工作队列，准备好再次由工作人员弹出并执行 - 换句话说，它看起来像工作线程角度的堆栈。当一个工作人员耗尽其工作队列时，它会四处走动并试图从其他工作队列中窃取任务。那是偷工作。

Since day one, ForkJoinPool had a work queue per worker thread (let's call them "worker queues"). The forked tasks are pushed into the local worker queue, ready to be popped by the worker again and be executed -- in other words, it looks like a stack from worker thread perspective. When a worker depletes its worker queue, it goes around and tries to steal the tasks from other worker queues. That is "work stealing".

现在，在（IIRC）JDK 7u12之前， ForkJoinPool 有一个全局提交队列。当工作线程用尽本地任务以及窃取任务时，他们到达那里并试图查看外部工作是否可用。在这种设计中，对于由 ArrayBlockingQueue 支持的常规，例如 ThreadPoolExecutor 没有任何优势。

Now, before (IIRC) JDK 7u12, ForkJoinPool had a single global submission queue. When worker threads ran out of local tasks, as well the tasks to steal, they got there and tried to see if external work is available. In this design, there is no advantage against a regular, say, ThreadPoolExecutor backed by ArrayBlockingQueue.

此后发生了重大变化。在此提交队列被确定为严重的性能瓶颈之后，Doug Lea等人。条纹提交队列也是如此。事后看来，这是一个明显的想法：您可以重用大多数可用于工作队列的机制。您甚至可以为每个工作线程松散地分发这些提交队列。现在，外部提交进入其中一个提交队列。然后，没有工作的工作人员可以首先查看与特定工作者关联的提交队列，然后四处寻找其他人的提交队列。人们也可以打电话给 偷窃。

It changed significantly after then. After this submission queue was identified as the serious performance bottleneck, Doug Lea et al. striped the submission queues as well. In hindsight, that is an obvious idea: you can reuse most of the mechanics available for worker queues. You can even loosely distribute these submission queues per worker threads. Now, the external submission goes into one of the submission queues. Then, workers that have no work to munch on, can first look into the submission queue associated with a particular worker, and then wander around looking into the submission queues of others. One can call that "work stealing" too.

我看到很多工作负载从中受益。很久以前就已经认识到 ForkJoinPool 的这种特殊设计优势，即使对于普通的非递归任务也是如此。 concurrency-interest @的许多用户都要求一个简单的工作窃取执行程序，而没有所有的 ForkJoinPool arcanery。这是原因之一，为什么我们 Executors.newWorkStealingPool（） - 目前委托给 ForkJoinPool ，但是打开以提供更简单的实现。

I have seen many workloads benefiting from this. This particular design advantage of ForkJoinPool even for plain non-recursive tasks was recognized a long ago. Many users at concurrency-interest@ asked for a simple work-stealing executor without all the ForkJoinPool arcanery. This is one of the reasons, why we have Executors.newWorkStealingPool() in JDK 8 onward -- currently delegating to ForkJoinPool, but open for providing a simpler implementation.

这篇关于Java ForkJoinPool具有非递归任务，是否可以正常工作？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！