如何正确并行运行多个异步任务？ [英] How to properly run multiple async tasks in parallel?

问题描述

如果你需要并行运行多个异步I / O任务，但需要确保不超过X I / O处理更多的是在同一时间运行的是什么;和pre和后I / O处理任务不应该有这样的限制。

What if you need to run multiple asynchronous I/O tasks in parallel but need to make sure that no more than X I/O processes are running at the same time; and pre and post I/O processing tasks shouldn't have such limitation.

下面是一个场景 - 假设有1000个任务;他们每个人的接收的文本串作为输入参数;转换文本（pre I / O处理），然后写道，转化成文本文件。我们的目标是使pre-处理逻辑利用CPU /内核的100％和I /任务Ø部分，最大运行10并行度（最大10同时打开在一个时间写文件）。

Here is a scenario - let's say there are 1000 tasks; each of them accepts a text string as an input parameter; transforms that text (pre I/O processing) then writes that transformed text into a file. The goal is to make pre-processing logic utilize 100% of CPU/Cores and I/O portion of the tasks run with max 10 degree of parallelism (max 10 simultaneously opened for writing files at a time).

您可以提供样本code如何用C＃做/ .NET 4.5？

Can you provide a sample code how to do it with C# / .NET 4.5?

<一个href=\"http://blogs.msdn.com/b/csharpfaq/archive/2012/01/23/using-async-for-file-access-alan-berman.aspx\">http://blogs.msdn.com/b/csharpfaq/archive/2012/01/23/using-async-for-file-access-alan-berman.aspx

推荐答案

我认为使用TPL数据流为这将是一个不错的主意：你创建无限并行pre型和后处理模块，文件写入有限的并行性，并阻止它们链接在一起。是这样的：

I think using TPL Dataflow for this would be a good idea: you create pre- and post-process blocks with unbounded parallelism, a file-writing block with limited parallelism and link them together. Something like:

var unboundedParallelismOptions =
    new ExecutionDataflowBlockOptions
    {
        MaxDegreeOfParallelism = DataflowBlockOptions.Unbounded
    };

var preProcessBlock = new TransformBlock<string, string>(
    s => PreProcess(s), unboundedParallelismOptions);

var writeToFileBlock = new TransformBlock<string, string>(
    async s =>
            {
                await WriteToFile(s);
                return s;
            },
    new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 10 });

var postProcessBlock = new ActionBlock<string>(
    s => PostProcess(s), unboundedParallelismOptions);

var propagateCompletionOptions =
    new DataflowLinkOptions { PropagateCompletion = true };

preProcessBlock.LinkTo(writeToFileBlock, propagateCompletionOptions);
writeToFileBlock.LinkTo(postProcessBlock, propagateCompletionOptions);

// use something like await preProcessBlock.SendAsync("text") here

preProcessBlock.Complete();
await postProcessBlock.Completion;

其中，将writeToFile（）可能看起来像这样：

private static async Task WriteToFile(string s)
{
    using (var writer = new StreamWriter(GetFileName()))
        await writer.WriteAsync(s);
}

这篇关于如何正确并行运行多个异步任务？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！