尝试并行运行多个 HTTP 请求，但受 Windows(注册表)限制 [英] Trying to run multiple HTTP requests in parallel, but being limited by Windows (registry)
问题描述
我正在开发一个应用程序(winforms C# .NET 4.0)，我通过一个简单的 HTTP 请求从第三方访问查找功能.我用一个参数调用一个 url，作为回报，我得到一个带有查找结果的小字符串.很简单.
I'm developing an application (winforms C# .NET 4.0) where I access a lookup functionality from a 3rd party through a simple HTTP request. I call an url with a parameter, and in return I get a small string with the result of the lookup. Simple enough.
然而，挑战在于，我必须进行大量此类查找(数千次)，并且我想限制所需的时间.因此，我想并行运行请求(比如 10-20).我使用 ThreadPool 来执行此操作，我的代码的简短版本如下所示:
The challenge is however, that I have to do lots of these lookups (a couple of thousands), and I would like to limit the time needed. Therefore I would like to run requests in parallel (say 10-20). I use a ThreadPool to do this, and the short version of my code looks like this:
public void startAsyncLookup(Action<LookupResult> returnLookupResult)
{
this.returnLookupResult = returnLookupResult;
foreach (string number in numbersToLookup)
{
ThreadPool.QueueUserWorkItem(lookupNumber, number);
}
}
public void lookupNumber(Object threadContext)
{
string numberToLookup = (string)threadContext;
string url = @"http://some.url.com/?number=" + numberToLookup;
WebClient webClient = new WebClient();
Stream responseData = webClient.OpenRead(url);
LookupResult lookupResult = parseLookupResult(responseData);
returnLookupResult(lookupResult);
}
我从另一个地方填写了numbersToLookup(一个List)，调用startAsyncLookup并为其提供回调函数returnLookupResult 返回每个结果.这有效，但我发现我没有得到我想要的吞吐量.
I fill up numbersToLookup (a List<String>) from another place, call startAsyncLookup and provide it with a call-back function returnLookupResult to return each result. This works, but I found that I'm not getting the throughput I want.
最初我认为这可能是第 3 方的系统很差，但我通过尝试同时从两台不同的机器运行相同的代码来排除这一点.两个人中的每一个都和一个人做的一样长，所以我可以排除那个.
Initially I thought it might be the 3rd party having a poor system on their end, but I excluded this by trying to run the same code from two different machines at the same time. Each of the two took as long as one did alone, so I could rule out that one.
然后一位同事提示我这可能是 Windows 中的一个限制.我用谷歌搜索了一下，发现其中有这篇文章默认情况下Windows 将同时向同一 Web 服务器的请求数量限制为 HTTP 1.0 为 4，HTTP 1.1 为 2(对于 HTTP 1.1，这实际上是根据规范 (RFC2068)).
A colleague then tipped me that this might be a limitation in Windows. I googled a bit, and found amongst others this post saying that by default Windows limits the number of simultaneous request to the same web server to 4 for HTTP 1.0 and to 2 for HTTP 1.1 (for HTTP 1.1 this is actually according to the specification (RFC2068)).
上面提到的同一篇文章也提供了一种增加这些限制的方法.通过向 [HKEY_CURRENT_USERSoftwareMicrosoftWindowsCurrentVersionInternet Settings](MaxConnectionsPerServer 和 MaxConnectionsPer1_0Server)添加两个注册表值，我可以自己控制.
The same post referred to above also provided a way to increase these limits. By adding two registry values to [HKEY_CURRENT_USERSoftwareMicrosoftWindowsCurrentVersionInternet Settings] (MaxConnectionsPerServer and MaxConnectionsPer1_0Server), I could control this myself.
所以，我尝试了这个(两个都设置为 20)，重新启动了我的计算机，并尝试再次运行我的程序.可悲的是，它似乎没有任何帮助.在运行批处理查找时，我还密切关注资源监视器，我注意到我的应用程序(标题涂黑的应用程序)仍然只使用两个 TCP 连接.
So, I tried this (sat both to 20), restarted my computer, and tried to run my program again. Sadly though, it didn't seem to help any. I also kept an eye on the Resource Monitor while running my batch lookup, and I noticed that my application (the one with the title blacked out) still only was using two TCP connections.
那么，问题是，为什么这不起作用?我链接的帖子是否使用了错误的注册表值?这也许不再可能在 Windows 中破解"(我使用的是 Windows 7)?
So, the question is, why isn't this working? Is the post I linked to using the wrong registry values? Is this perhaps not possible to "hack" in Windows any longer (I'm on Windows 7)?
以防万一有人想知道，我还尝试了对 ThreadPool 上 MaxThreads 的不同设置(从 10 到 100 的所有设置)，这似乎根本不影响我的吞吐量，所以问题不应该是也有.
And just in case anyone should wonder, I have also tried with different settings for MaxThreads on ThreadPool (everything from 10 to 100), and this didn't seem to affect my throughput at all, so the problem shouldn't be there either.
推荐答案
服务点.它为 HTTP 连接提供连接管理.ServicePoint 对象允许的默认最大并发连接数为 2.因此，如果您需要增加它，您可以使用 ServicePointManager.DefaultConnectionLimit 属性.只需查看 MSDN 中的链接，您就可以看到示例.并设置您需要的值.
It is matter of ServicePoint. Which provides connection management for HTTP connections. The default maximum number of concurrent connections allowed by a ServicePoint object is 2. So if you need to increase it you can use ServicePointManager.DefaultConnectionLimit property. Just check the link in MSDN there you can see a sample. And set the value you need.
这篇关于尝试并行运行多个 HTTP 请求，但受 Windows(注册表)限制的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
