该操作已通过WebClient.DownloadFile和正确的url超时 [英] The operation has timed out with WebClient.DownloadFile and correct url's

问题描述

我正在将产品批量上传到数据库。

I am batch uploading products to a database.

我正在将图像URL下载到用于产品的站点。

I am download the image urls to the site to be used for the products.

我编写的代码在前25次迭代中工作正常（出于某种原因始终使用该数字），但是随后抛出System.Net.WebException操作已超时。

The code I written works fine for the first 25 iterations (always that number for some reason), but then throws me a System.Net.WebException "The operation has timed out".

if (!File.Exists(localFilename))
{
    using (WebClient Client = new WebClient())
    {
        Client.DownloadFile(remoteFilename, localFilename);
    }
}

我检查了它正在请求的远程URL一个有效的图像URL会返回一个图像。

I checked the remote url it was requesting and it is a valid image url that returns an image.

此外，当我使用调试器逐步浏览它时，没有超时错误。

Also, when I step through it with the debugger, I don't get the timeout error.

帮助！ ;）

推荐答案

如果我穿上你的鞋子，可以尝试以下几种方法：

If I were in your shoes, here's a few possibilities I'd investigate:

• 如果从多个线程运行此代码，则可能会遇到System.Net.ServicePointManager.DefaultConnectionLimit属性。启动应用程序时，尝试将其增加到50-100。请注意，我认为这不是您的问题，但尝试进行此操作比下面的其他操作容易。 ：-）


• 另一种可能性是您要淹没服务器。通常，使用单线程客户端很难做到这一点，但由于多个其他客户端也可能访问服务器，因此这是可能的。但是，由于问题总是发生在＃25，因此似乎不太可能，因为您希望看到更多变化。


• 您可能在客户端与服务器之间备份keepalive HTTP连接时遇到问题。这似乎也不太可能。


• 硬性限制25使得我认为这可能是代理限制或防火墙限制，无论是在您端还是服务器端，从一个客户端IP到一台服务器的连接数都超过25（

• if you're running this code from multiple threads, you may be bumping up against the System.Net.ServicePointManager.DefaultConnectionLimit property. Try increasing it to 50-100 when you start up your app. note that I don't think this is your problem, but trying this is easier than the other stuff below. :-)
• another possibility is that you're swamping the server. This is usually hard to do with a single-threaded client, but is possible since multiple other clients may be hitting the server also. But because the problem always happens at #25, this seems unlikely since you'd expect to see more variation.
• you may be running into a problem with keepalive HTTP connections backing up between your client and the server. this also seems unlikely.
• the hard cutoff of 25 makes me think that this may be a proxy or firewall limit, either on your end or the server's, where >25 connections made from one client IP to one server (or proxy) will get throttled.

我的钱在后一个，因为它总是在一个很好的整数处折断。的请求，而进入调试器（又是更慢！）不会触发问题。

My money is on the latter one, since the fact that it always breaks at a nice round number of requests, and that stepping in the debugger (aka slower!) doesn't trigger the problem.

要测试所有这些，我将从简单的事情开始：在每个HTTP调用之前延迟（Thread.Sleep），看看问题是否消失。如果是这样，请减少延迟，直到问题再次出现。如果不是这样，请增加大量的延迟（例如10秒），直到问题消失。如果延迟10秒仍未消失，那确实是个谜，我需要更多信息来进行诊断。

To test all this, I'd start with the easy thing: stick in a delay (Thread.Sleep) before each HTTP call, and see if the problem goes away. If it does, reduce the delay until the problem comes back. If it doesn't, increase the delay up to a large number (e.g. 10 seconds) until the problem goes away. If it doesn't go away with a 10 second delay, that's truly a mystery and I'd need more info to diagnose.

如果延迟消失了，那么您需要弄清楚原因-以及该限制是否永久（例如，您无法更改的服务器防火墙）或您可以更改的内容。要获取更多信息，您需要对请求进行计时（例如，在每次调用之前和之后检查DateTime.Now），以查看是否有模式。如果时间一致并且突然变大，则表明网络/防火墙/代理节流。如果时间逐渐增加，则表明您正在逐渐使服务器超载并延长其请求队列。

If it does go away with a delay, then you need to figure out why-- and whether the limit is permanent (e.g. server's firewall which you can't change) or something you can change. To get more info, you'll want to time the requests (e.g. check DateTime.Now before and after each call) to see if you see a pattern. If the timings are all consistent and suddenly get huge, that suggests a network/firewall/proxy throttling. If the timings gradually increase, that suggests a server you're gradually overloading and lengthening its request queue.

除了对请求进行计时之外，我还将您的webclient调用的超时设置为更长，这样您就可以确定超时是无限的还是只是一点点长于默认值。为此，您需要WebClient类的替代方法，因为它不支持超时。 此线程 MSDN论坛有一个合理的替代代码示例。

In addition to timing the requests, I'd set the timeout of your webclient calls to be longer, so you can figure out if the timeout is infinite or just a bit longer than the default. To do this, you'll need an alternative to the WebClient class, since it doesn't support a timeout. This thread on MSDN Forums has a reasonable alternative code sample.

在代码中添加计时的另一种方法是使用Fiddler：

An alternative to adding timing in your code is to use Fiddler:

• 下载小提琴手并启动它。


• 设置您的网络客户端代码的Proxy属性，使其指向提琴手代理（localhost：8888）


• 运行您的应用并查看提琴手。

• download fiddler and start it up.
• set your webclient code's Proxy property to point to the fiddler proxy (localhost:8888)
• run your app and look at fiddler.

这篇关于该操作已通过WebClient.DownloadFile和正确的url超时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！