IO绑定操作的并行执行 [英] Parallel execution for IO bound operations

问题描述

我已经阅读了TPL和任务库文档的封面。但是，我仍然无法很清楚地理解以下情况，现在我需要实施它。

I have read TPL and Task library documents cover to cover. But, I still couldn't comprehend the following case very clearly and right now I need to implement it.

我将简化我的情况。我有一个长度为1000的 IEnumerable< Uri> 。我必须使用 HttpClient 对其进行请求。

I will simplify my situation. I have an IEnumerable<Uri> of length 1000. I have to make a request for them using HttpClient.

我有两个问题。

1. 计算量不大，只需等待Http请求即可。在这种情况下，我仍然可以使用 Parallel.Foreach（）吗？


2. 在使用 Task的情况下，创建大量的最佳实践是什么？假设我使用 Task.Factory.StartNew（）并将这些任务添加到列表中，然后等待所有任务。是否有功能（例如TPL分区程序）控制我可以创建的最大任务数和最大 HttpClient ？

1. There is not much computation, just waiting for Http request. In this case can I still use Parallel.Foreach() ?
2. In case of using Task instead, what is the best practice for creating huge number of them? Let's say I use Task.Factory.StartNew() and add those tasks to a list and wait for all of them. Is there a feature (such as TPL partitioner) that controls number of maximum tasks and maximum HttpClient I can create?

SO上有几个类似的问题，但是没有人提到最大数量。要求只是使用具有最大HttpClient的最大任务。

There are couple of similar questions on SO, but no one mentions the maximums. The requirement is just using maximum tasks with maximum HttpClient.

预先感谢您。

推荐答案

在这种情况下，我仍然可以使用 Parallel.Foreach 吗？

这确实不合适。 Parallel.Foreach 对于CPU密集型工作而言更为有效。它还不支持异步操作。

This isn't really appropriate. Parallel.Foreach is more for CPU intensive work. It also doesn't support async operations.

如果使用 Task ，创建大量的最佳做法是什么？

In case of using Task instead, what is the best practice for creating huge number of them?

使用TPL Dataflow块代替。您不会在等待线程可用的情况下创建大量任务。您可以配置最大任务数，并将它们重用于所有同时位于等待任务的缓冲区中的项目。例如：

Use a TPL Dataflow block instead. You don't create huge amounts of tasks that sit there waiting for a thread to become available. You can configure the max amount of tasks and reuse them for all the items that meanwhile sit in a buffer waiting for a task. For example:

var block = new ActionBlock<Uri>(
    uri => SendRequestAsync(uri),
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 50 });

foreach (var uri in uris)
{
    block.Post(uri);
}

block.Complete();
await block.Completion;

这篇关于IO绑定操作的并行执行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！