用于错误处理的并行I / O和重试逻辑 [英] Parallel I/O and Retry Logic for Error Handling

问题描述

通常并行处理仅与CPU密集型操作相关。然而，PLINQ特别使用WithDegreeOfParallelism扩展来提供IO密集型支持。例如：

 来自网站的新的[] 
 {
www.albahari.com 
www.linqpad.net，
www.oreilly.com，
www.takeonit.com，
stackoverflow.com，
 www.rebeccarey.com
} 
 .AsParallel（）。WithDegreeOfParallelism（6）
 let p = new Ping（）。发送（网站）
选择新
 {
 site，
 Result = p.Status，
 Time = p.RoundtripTime 
} 
  

但是如果支持IO是WithDegreeOfParallelism的目标，PLINQ如何进一步扩展或用于实现重试效应，这是IO操作的典型？那么延迟效应呢？

例如，如果IO通过WCF服务调用引发了一个CommunicationException，我可能希望再次使用3尝试策略再次提出相同的请求，然后再转到下一个资源。我也可能想在每次尝试之间等一下。而且，在每次尝试之间等待一分钟时，我不希望线程阻塞等待。

在这个 MSDN文章，作者以类似于上面显示的查询开始。然后他转换查询，使得没有线程被阻止。不幸的是，他在过程中失去了WithDegreeOfParallelism。无论哪种方式，他发生错误时都没有解决重试的问题。

任何人都有或知道这样做的光滑方式？

我正在考虑制作一个特殊的IEnumerable包装器，允许在PLINQ进行集合时重新插入值。这确实会导致重试行为，但仍然不允许重试之间延迟1分钟。我可以用Thread.Sleep（）创建1分钟的延迟，但是我试图不阻止线程。

想法？

解决方案

文章你 linked 实际上显示了如何通过使用 Task 的API（ DownloadDataTask ）而是：

然而，仍然有一些关于这段代码不理想的东西。
工作（发送下载请求和阻止）需要几乎
没有CPU，但是它正在由ThreadPool线程完成，因为我使用
默认调度程序。理想情况下，线程只能用于CPU限制的
工作（当有实际工作时）。

使用PLINK / 对于基于IO的操作的Task.Run / Task.Factory.StartNew 是一个反模式即可。 PLINQ（与 Parallel.For 等一样）对于CPU绑定的计算工作是有好处的，但是对于自然异步网络/ IO-绑定操作，在飞行中中根本不需要线程。要遵循您显示的示例代码，那将像 new Ping（）。SendAsync（site），返回一个任务。然后，您可以执行等待Task.WhenAll（tasks）并处理错误。

请参阅没有线程由Stephen Cleary和他最近的<一个href =https://stackoverflow.com/a/22232406/1768303> answer 解决并行IO的最大程度。除此之外，很容易纳入重试逻辑，而不会涉及任何线程（例如此） 。

Normally parallel processing is relevant only to CPU intensive operations. However, PLINQ specifically provides IO-intensive support using the WithDegreeOfParallelism extension. For example:

from site in new[]
{
    "www.albahari.com",
    "www.linqpad.net",
    "www.oreilly.com",
    "www.takeonit.com",
    "stackoverflow.com",
    "www.rebeccarey.com"  
}
.AsParallel().WithDegreeOfParallelism(6)
let p = new Ping().Send (site)
select new
{
    site,
    Result = p.Status,
    Time = p.RoundtripTime
}

But if supporting IO is the goal of WithDegreeOfParallelism, how then can PLINQ be further extended or used to achieve a "retry" effect, which is typical of IO operations? And what about a "delay" effect?

For example, if IO through a WCF service call throws a CommunicationException, I might want the same request made again with a "3 tries" strategy before moving on to the next resource. I might also want a minute wait between each try. And while I "wait" for a minute between each try, I don't want a thread blocked waiting.

In this MSDN article the author starts with a query similar to what I've shown above. He then transforms the query so that no threads are blocking. Unfortunately he lost the WithDegreeOfParallelism in the process. And either way, he did not address the issue of "retries" when an error occurs.

Anyone have or know of a slick way of doing this?

I was thinking of making a special IEnumerable wrapper that permitted values to be "re-inserted" while the collection was being walked by PLINQ. This would indeed cause a "retry" behavior, but it would still not allow for the "1 minute delay between retries" requirement. I could create the 1 minute delay with Thread.Sleep(), but I'm trying not to block threads.

Thoughts?

解决方案

The article you linked actually shows how to avoid blocking threads for IO-bound operations by using a Task-based API (DownloadDataTask) instead:

However, there’s still something about this code that is not ideal. The work (sending off download requests and blocking) requires almost no CPU, but it is being done by ThreadPool threads since I’m using the default scheduler. Ideally, threads should only be used for CPU-bound work (when there’s actually work to do).

Using PLINK / Task.Run/ Task.Factory.StartNew for IO-based operations is an anti-pattern. PLINQ (same as Parallel.For etc) is good for CPU-bound computational work, but there is no point in allocating and blocking a thread for a naturally asynchronous network/IO-bound operation, which doesn't need a thread at all while "in-flight". To follow the sample code you showed, that would be something like new Ping().SendAsync(site), returning a Task. You could then do await Task.WhenAll(tasks) and process the errors.

Refer to "There Is No Thread" by Stephen Cleary, and his recent answer addressing the max degree of parallel IO. On top of that, it's quite easy to incorporate a retry logic, without getting any threads involved (for example, like this).

这篇关于用于错误处理的并行I / O和重试逻辑的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！