扩展多个HttpWebRequests? [英] Scaling up Multiple HttpWebRequests?

问题描述

我正在构建一个服务器应用程序，该应用程序需要持续对其他几个服务器执行许多http请求.目前，我基本上设置了大约30个线程，并在每个线程上连续不断地运行HttpWebRequests，从而实现了每秒大约30个请求的吞吐量.

I'm building a server application that needs to perform a lot of http requests to a couple other servers on an ongoing basis. Currently, I'm basically setting up about 30 threads and continuously running HttpWebRequests synchronously on each thread, achieving a throughput of about 30 requests per second.

我确实在app.config中设置了ServicePoint ConnectionLimit，所以这不是限制因素.

I am indeed setting the ServicePoint ConnectionLimit in the app.config so that's not the limiting factor.

我需要大幅度扩大规模.至少我需要更多的CPU能力，但是我想知道我是否可以通过使用HttpWebRequest对象的异步方法(例如.BeginGetResponse())获得任何好处，而不是自己创建线程并使用这些线程上的同步方法(例如:.GetResponse()).

I need to scale this up drastically. At the very least I'll need some more CPU horse power, but I'm wondering if I would gain any advantages by using the async methods of the HttpWebRequest object (eg: .BeginGetResponse() ) as opposed to creating threads myself and using the synchronous methods (eg: .GetResponse() ) on these threads.

如果我使用异步方法，显然我必须大幅重新设计我的应用程序，因此我想知道是否有人在我重新编码所有内容之前可能有一些见识，以防万一我要吃午饭了.

If I go with the async methods, I obviously have to significantly redesign my app, so I'm wondering if anyone might have some insight before I go and recode everything, in case I'm out to lunch.

谢谢！

推荐答案

如果您使用的是Windows NT，则System.Net.Sockets.Socket类始终将IO完成端口用于异步操作.异步模式下的HTTPWebRequest使用异步套接字，因此将使用IOCP.

If you are on Windows NT, then System.Net.Sockets.Socket class always uses IO Completion ports for async operations. And HTTPWebRequest in async mode uses async sockets, and hence will be using IOCP.

如果不进行详细的基准测试，则很难说我们的瓶颈是在应用程序中还是在服务器的远程方面，是在HttpWebRequest内部还是在堆栈中.但是，可以肯定地说，asyncc会给您带来更好的性能，因为它最终将在后台使用IOCP.重新实现异步应用并不困难.

Without doing detailed benchmarking, it is difficult to say if our bottleneck is inside HttpWebRequest, or up the stack, in your application, or on the remote side, in the server. But offhand, for sure, asyncc will give you better performance, because it will end up using IOCP under the covers. And reimplementing the app for async is not that difficult.

因此，我建议您首先将应用程序体系结构更改为异步.然后查看您获得了多少最大吞吐量.然后，您可以开始进行基准测试，找出瓶颈所在，然后消除瓶颈.

So, I would suggest that you first change your app architecture to async. Then see how much max throughput you are getting. Then you can start benchmarking and finding out where the bottleneck is, and removing that.

这篇关于扩展多个HttpWebRequests?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！