超时支持的多线程Web请求 [英] Timeout supported Multithreaded Web Request

问题描述

如果将其用于简单目的，则WebClient就足够了.缺少超时，您可以继承它并轻松地重写它的GetWebRequest方法:

If you are using it for simple purposes, WebClient is enough. Absence of Timeout you can inherit it and override it's GetWebRequest method easily:

protected override WebRequest GetWebRequest(Uri address)
{
    WebRequest request = base.GetWebRequest(address);
    request.Timeout = Timeout;
    return request;
}

如果使用多线程方式，则必须通过代码或app.config设置MaxConnection:

If you are using it multithreaded way, you have to set MaxConnection by code or app.config: Improving performance of multithreaded HttpWebRequests in .NET

您还可以轻松并行化它: 在.net 4.0中并行化Web爬网程序的最佳实践

You can parallelize it easily also: Best practics for parallelize web crawler in .net 4.0

也许微软为WebClient(HttpWebRequest)的实现问题创建了新的HttpClient类

Maybe Microsoft created new HttpClient class for implemantation problems of WebClient (HttpWebRequest)

但是它没有Proxy，Gzip支持.

But it does not have Proxy,Gzip support.

对于WebClient:

For WebClient:

设置超时在多线程实现中仍然无用！

Setting up Timeout still useless in multithreaded implemantation!

HttpWebRequest上的并发限制

那么问题是

我需要具有代理，超时(工作正常)，gzip支持的FastWebClient.我将它与多个代理一起使用，因此我肯定需要多个FastWebClient.

I need FastWebClient with proxy,timeout(working one),gzip support. I will use it with multiple proxies, so I definetely need multiple FastWebClient.

或助手:

FastWebHelper.DownloadString(字符串url，WebProxy代理，int超时)

FastWebHelper.DownloadString (string url,WebProxy proxy,int timeout)

有什么主意吗?

推荐答案

好的. WebClient.DownloadStringAsync方法 您仍然需要为每个连接/请求创建1个WebClient实例，因为它不是无状态的.

Sure. WebClient.DownloadStringAsync Method You would still need to create 1 WebClient instance per connection/request, as it's not stateless.

这篇关于超时支持的多线程Web请求的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！