ForkJoinPool在invokeAll/join期间停顿 [英] ForkJoinPool stalls during invokeAll/join

问题描述

我尝试使用 ForkJoinPool 并行化我的CPU密集型计算. 我对ForkJoinPool的理解是，只要可以执行任何任务，它就可以继续工作.不幸的是，我经常观察到工作线程空闲/正在等待，因此并非所有CPU都保持忙碌状态.有时，我什至还观察到了其他工作线程.

I try to use a ForkJoinPool to parallelize my CPU intensive calculations. My understanding of a ForkJoinPool is, that it continues to work as long as any task is available to be executed. Unfortunately I frequently observed worker threads idling/waiting, thus not all CPU are kept busy. Sometimes I even observed additional worker threads.

我没有想到这一点，因为我严格尝试使用非阻止任务. 我的观察与 ForkJoinPool似乎浪费线程非常相似. 在对ForkJoinPool进行了大量调试之后，我有了一个猜测:

I did not expect this, as I strictly tried to use non blocking tasks. My observation is very similar to those of ForkJoinPool seems to waste a thread. After debugging a lot into ForkJoinPool I have a guess:

我使用invokeAll()在子任务列表上分配工作. invokeAll()完成执行第一个任务本身后，便开始加入其他任务.这可以正常工作，直到要加入的下一个任务位于执行队列的顶部.不幸的是，我异步提交了其他任务，而没有加入它们.我希望ForkJoin框架能够首先继续执行那些任务，而不是转而加入其余的任务.

I used invokeAll() to distribute work over a list of subtasks. After invokeAll() finished to execute the first task itself it starts joining the other ones. This works fine, until the next task to join is on top of the executing queue. Unfortunately I submitted additional tasks asynchronously without joining them. I expected the ForkJoin framework to continue executing those task first and than turn back to joining any remaining tasks.

但似乎无法通过这种方式工作.取而代之的是，工作线程会因调用wait()而停滞，直到等待任务的任务准备就绪(大概由另一个工作线程执行).我没有对此进行验证，但这似乎是调用join()的一个普遍缺陷.

But it seems not to work this way. Instead the worker thread gets stalled calling wait() until the task waiting for gets ready (presumably executed by an other worker thread). I did not verify this, but it seems to be a general flaw of calling join().

ForkJoinPool提供了

ForkJoinPool provides an asyncMode, but this is a global parameter and can not be used for individual submissions. But I like to see my asynchronously forked tasks to be executed soon.

那么，为什么ForkJoinTask.doJoin()不能简单地在队列准备好之前执行任何可用的任务，直到它准备好(自己执行或被他人偷走)?

So, why does ForkJoinTask.doJoin() not simply executes any available task on top of its queue until it gets ready (either executed by itself or stolen by others)?

推荐答案

由于似乎没有其他人理解我的问题，因此我尝试解释经过一整夜的调试后发现的问题:

Since nobody else seems to understand my question I try to explain what I found after some nights of debugging:

如果所有fork/join调用都严格配对，则ForkJoinTasks的当前实现效果很好.通过一个开括号来说明一个叉子，然后通过一个闭括号来说明一个完美的二元叉子联接模式，如下所示:

The current implementation of ForkJoinTasks works well if all fork/join calls are strictly paired. Illustrating a fork by an opening bracket and join by a closing one a perfect binary fork join pattern may look like this:

{([] [])([] [])} {([[[[])([] [])}}

如果使用invokeAll()，您还可以提交子任务列表，如下所示:

If you use invokeAll() you may also submit list of subtasks like this:

{([] [] [] [])([] [] [] [])([] [] [] [])}

但是我所做的看起来像这样:

What I did however looks like this pattern:

{([)([)} ...]]

您可能会认为这看起来很糟糕，或者是对fork-join框架的滥用.但是唯一的限制是，任务完成依赖项是非循环的，否则您可能会陷入僵局.只要我的 [] 任务不依赖于()任务，我就看不到任何问题.令人反感的]] 只是表示我不明确地等待它们；他们可能会结束一天，对我而言(那时)无关紧要.

You may argue this looks ill or is a misuse of the fork-join framework. But the only constraint is, that the tasks completion dependencies are acyclic, else you may run into a deadlock. As long as my [] tasks are not dependent on the () tasks, I don't see any problem with it. The offending ]]'s just express that I do not wait for them explicitly; they may finish some day, it does not matter to me (at that point).

实际上，当前的实现能够执行我的互锁任务，但只能通过生成效率很低的附加帮助程序线程来实现.

Indeed the current implementation is able to execute my interlocked tasks, but only by spawning additional helper threads which is quite inefficient.

该缺陷似乎是join()的当前实现:加入)希望在其执行队列的顶部看到其相应的()，但是它找到了一个 [并且感到困惑.不是简单地执行 [] 来摆脱它，而是当前线程挂起(调用wait())，直到其他人来执行意外操作任务.这会导致性能急剧下降.

The flaw seems to be the current implementation of join(): joining an ) expects to see its corresponding ( on top of its execution queue, but it finds a [ and is perplexed. Instead of simply executing [] to get rid of it, the current thread suspends (calling wait()) until someone else comes around to execute the unexpected task. This causes a drastic performance break down.

我的主要目的是在队列上进行其他工作，以防止工作线程在队列为空时挂起.不幸的是，情况恰好相反:-(

My primary intend was to put additional work onto the queue to prevent the worker thread from suspending if the queue runs empty. Unfortunately the opposite happens :-(

这篇关于ForkJoinPool在invokeAll/join期间停顿的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！