WhenAll VS为WaitAll并行 [英] WhenAll vs WaitAll in parallel
问题描述
我想了解如何为WaitAll 和 WhenAll 的作品，并有以下的问题。有两种可能的方式来获得从方法的结果是：
I'm trying to understand how WaitAll and WhenAll works and have following problem. There are two possible ways to get a result from a method:
-
返回Task.WhenAll（任务）.Result.SelectMany（R =＆GT; R）; -
返回tasks.Select（T =＆GT; t.Result）.SelectMany（R =＆GT; R）.ToArray（）;
return Task.WhenAll(tasks).Result.SelectMany(r=> r);return tasks.Select(t => t.Result).SelectMany(r => r).ToArray();
如果我理解正确的话，第二种情况是像调用为WaitAll 在工作和提取后的结果。
If I understand correctly, the second case is like calling WaitAll on tasks and fetching the results after that.
它看起来像第二种情况下具有更好的性能。我知道， WhenAll 的正确用法是等待关键字，不过，我不知道为什么有这么在这些线路的性能相差很大。
It looks like the second case has much better performance. I know that the proper usage of WhenAll is with await keyword, but still, i'm wondering why there is so big difference in performance for these lines.
分析系统，我想我已经找到了如何的问题在一个简单的测试应用程序模型（测试code基于I3arnon回答）的流量后：
After analyzing the flow of the system I think I've figured out how to model the problem in a simple test application (test code is based on I3arnon answer):
public static void Test()
{
var tasks = Enumerable.Range(1, 1000).Select(n => Task.Run(() => Compute(n)));
var baseTasks = new Task[100];
var stopwatch = Stopwatch.StartNew();
for (int i = 0; i < 100; i++)
{
baseTasks[i] = Task.Run(() =>
{
tasks.Select(t => t.Result).SelectMany(r => r).ToList();
});
}
Task.WaitAll(baseTasks);
Console.WriteLine("Select - {0}", stopwatch.Elapsed);
baseTasks = new Task[100];
stopwatch.Restart();
for (int i = 0; i < 100; i++)
{
baseTasks[i] = Task.Run(() =>
{
Task.WhenAll(tasks).Result.SelectMany(result => result).ToList();
});
}
Task.WaitAll(baseTasks);
Console.WriteLine("Task.WhenAll - {0}", stopwatch.Elapsed);
}
看起来问题出在开始从其他任务（或并行循环）的任务。在这种情况下， WhenAll 结果在节目中更糟糕的表现。这是为什么呢？
It looks like the problem is in starting tasks from other tasks (or in Parallel loop). In that case WhenAll results in much worse performance of the program. Why is that?
推荐答案
您正在开始一个 Parallel.ForEach 循环，你应该避免内部任务。 Paralle.ForEach 的整点是在可用的CPU内核并行处理很多小而密集计算和启动任务并不是一个密集的计算。相反，它创建了一个任务，对象，并将其存储在一个队列如果任务池饱和，它很快将与1000任务被starteed。所以，现在 Parallel.ForEach 的任务池计算资源的竞争。
You are starting tasks inside a Parallel.ForEach loop which you should avoid. The whole point of Paralle.ForEach is to parallelize many small but intensive computations across the available CPU cores and starting a task is not an intensive computation. Rather it creates a task object and stores it on a queue if the task pool is saturated which it quickly will be with 1000 tasks being starteed. So now Parallel.ForEach competes with the task pool for compute resources.
在第一个循环是似乎调度是次优的，很少的CPU时，可能是因为 Task.WhenAll 在 Parallel.ForEach 来一个正常的循环，你会看到一个加速。
In the first loop that is quite slow it seems that the scheduling is suboptimal and very little CPU is used probably because of Task.WhenAll inside the Parallel.ForEach. If you change the Parallel.ForEach to a normal for loop you will see a speedup.
但是，如果你code没有任何状态结转迭代，你可以摆脱的任务，只需使用<$之间真的很简单，只要一个计算功能C $ C> Parallel.ForEach 来最大限度地提高性能：
But if you code really is as simple as a Compute function without any state carried forward between iterations you can get rid of the tasks and simply use Parallel.ForEach to maximize performance:
Parallel.For(0, 100, (i, s) =>
{
Enumerable.Range(1, 1000).Select(n => Compute(n)).SelectMany(r => r).ToList();
});
至于为什么 Task.WhenAll 执行更糟糕，你应该认识到，这code
As to why Task.WhenAll performs much worse you should realize that this code
tasks.Select(t => t.Result).SelectMany(r => r).ToList();
不会并行运行的任务。该了ToList 基本上包装了迭代的的foreach 循环和循环体创建一个任务，然后等待任务完成，因为您检索 Task.Result 属性。因此，循环的每次迭代将创建一个任务，然后等待它完成。 1000任务的执行中一个接一个地并且在处理任务很少的开销。这意味着你不需要的任务，这也是我所建议的上面。
will not run the tasks in parallel. The ToList basically wraps the iteration in a foreach loop and the body of the loop creates a task and then waits for the task to complete because you retrieve the Task.Result property. So each iteration of the loop will create a task and then wait for it to complete. The 1000 tasks are executed one after the other and there is very little overhead in handling the tasks. This means that you do not need the tasks which is also what I have suggested above.
在另一方面，在code
On the other hand, the code
Task.WhenAll(tasks).Result.SelectMany(result => result).ToList();
将启动所有任务，并尝试同时执行它们，因为任务池无法并行的大部分任务执行1000任务执行前排队。这就形成了一个大的管理和任务切换开销，这解释了糟糕的表现。
will start all the tasks and try to execute them concurrently and because the task pool is unable to execute 1000 tasks in parallel most of these tasks are queued before they are executed. This creates a big management and task switch overhead which explains the bad performance.
对于您添加的最后一个问题：如果在外工作的唯一目的是启动内部任务，那么外任务没有用处，但如果外部任务在那里执行某种协调内部的任务那么它可能是有意义的（也许你想要 Task.WhenAny 结合 Task.WhenAll ）。如果没有更多的情况下是很难回答。然而，你的问题似乎是有关性能和启动10万的任务可能会增加相当大的开销。
With regard to the final question you added: If the only purpose of the outer task is to start the inner tasks then the outer task has no useful purpose but if the outer tasks are there to perform some kind of coordination of the inner tasks then it might make sense (perhaps you want to combine Task.WhenAny with Task.WhenAll). Without more context it is hard to answer. However, your question seems to be about performance and starting 100,000 tasks may add considerable overhead.
Parallel.ForEach 是一个不错的选择，如果你要执行10万独立计算就像你在你的例子做。任务是很好的执行涉及到慢呼吁要等待和与结果相结合，并处理错误其它系统并发活动。对于大规模并行他们很可能不是最好的选择。
Parallel.ForEach is a good choice if you want to perform 100,000 independent computations like you do in your example. Tasks are very good for executing concurrent activities involving "slow" calls to other systems where you want to wait for and combine results and also handle errors. For massive parallelism they are probably not the best choice.
这篇关于WhenAll VS为WaitAll并行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
