如何在 F# Async.Parallel 中处理 HttpWebRequest 超时 [英] How to handle HttpWebRequest timeout in F# Async.Parallel
问题描述
我花了很长时间才弄明白为什么这段代码对某些网址挂起"了:
I just spent a long time breaking my teeth on why this code was 'hanging' for some urls:
let getImage (imageUrl:string) =
async {
try
let req = WebRequest.Create(imageUrl) :?> HttpWebRequest
req.UserAgent <- "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)";
req.Method <- "GET";
req.AllowAutoRedirect <- true;
req.MaximumAutomaticRedirections <- 4;
req.Timeout <- 3000; //HAHAHA, nice try!
let! response1 = req.AsyncGetResponse()
let response = response1 :?> HttpWebResponse
use stream = response.GetResponseStream()
let ms = new MemoryStream()
let bytesRead = ref 1
let buffer = Array.create 0x1000 0uy
while !bytesRead > 0 do
bytesRead := stream.Read(buffer, 0, buffer.Length)
ms.Write(buffer, 0, !bytesRead)
return SuccessfulDownload(imageUrl, ms.ToArray())
with
ex -> return FailedDownload(imageUrl, ex.Message)
}
在设法追踪 3000 个 url 中的哪一个挂起后,我了解到 AsyncGetResponse
不会注意到 HttpWebRequest.Timeout
.我做了一些搜索,其中提出了将异步请求包装在一个带有计时器的线程中的建议.这对 C# 来说很棒,但是如果我通过 Async.Parallel |> 运行其中的 3000 个Async.RunSynchronously
,处理这个问题的最佳方法是什么?
After managing to track down which of the 3000 urls was hanging, I learned that AsyncGetResponse
doesn't take any notice of HttpWebRequest.Timeout
. I've done a bit of searching which throws up suggestions of wrapping the async request in a thread with a timer. That's great for C#, but if I'm running 3000 of these through Async.Parallel |> Async.RunSynchronously
, what's the best way to handle this problem?
推荐答案
我只是粗略地测试了这个,但它应该有正确的行为:
I've only roughly tested this, but it should have the correct behavior:
type System.Net.WebRequest with
member req.AsyncGetResponseWithTimeout () =
let impl = async {
let iar = req.BeginGetResponse (null, null)
let! success = Async.AwaitIAsyncResult (iar, req.Timeout)
return if success then req.EndGetResponse iar
else req.Abort ()
raise (System.Net.WebException "The operation has timed out") }
Async.TryCancelled (impl, fun _ -> req.Abort ())
在您的代码中,调用 req.AsyncGetResponseWithTimeout()
而不是 req.AsyncGetResponse()
.
In your code, call req.AsyncGetResponseWithTimeout()
instead of req.AsyncGetResponse()
.
这篇关于如何在 F# Async.Parallel 中处理 HttpWebRequest 超时的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!