HttpWebRequest如何处理(过早)关闭基础TCP连接? [英] HttpWebRequest How to handle (premature) closure of underlying TCP connection?
问题描述
当使用.NET的HttpWebRequest类调用远程服务器(特别是REST Web服务)时，我很难弄清楚是否有一种方法可以处理潜在的连接问题.根据我的调查，WebClient类的行为是相同的，这在某种程度上是可以预料的，因为它似乎只提供了一个到HttpWebRequest的更简单的接口.
I have a hard time figuring out if there is a way to handle potential connectivity problems when using .NET's HttpWebRequest class to call a remote server (specifically a REST web service). From my investigations the behaviour of the WebClient class is the same, which is somewhat expected since it appears to only offer a more simple interface to the HttpWebRequest.
出于仿真目的，我编写了一个非常简单的HTTP服务器，该服务器的行为不符合HTTP 1.1 RFC.它的作用是接受客户端连接，然后发送适当的HTTP 1.1标头和"Hello World！".有效负载返回到客户端并关闭套接字，服务器端接受客户端连接的线程如下所示:
For simulation purposes, I've written a very simple HTTP server that does not behave according to the HTTP 1.1 RFC. What it does is it accepts a client connection, then sends appropriate HTTP 1.1 headers and a "Hello World!" payload back to the client and closes the socket, the thread accepting client connections on the server side looks as follows:
private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
private void Listen()
{
while (true)
{
using (TcpClient clientConnection = m_listener.AcceptTcpClient())
{
NetworkStream stream = clientConnection.GetStream();
StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
httpData.AppendFormat(m_defaultResponse);
Thread.Sleep(3000); // Sleep to simulate latency
stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);
stream.Close();
clientConnection.Close();
}
}
}
由于HTTP 1.1 RFC声明默认情况下HTTP 1.1使连接保持活动状态，并且服务器如果要关闭连接，则必须发送"Connection:Close"响应标头，这对于客户端而言是意外行为.客户端通过以下方式使用HttpWebRequest:
Since the HTTP 1.1 RFC states that HTTP 1.1 by default keeps connections alive and that a server must send a "Connection: Close" response header if it wants to close a connection this is unexpected behaviour for the client-side. The client uses HttpWebRequest in the following way:
private static void SendRequest(object _state)
{
WebResponse resp = null;
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
request.Timeout = 50 * 1000;
DateTime requestStart = DateTime.Now;
resp = request.GetResponse();
TimeSpan requestDuration = DateTime.Now - requestStart;
Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.Timeout)
{
Console.WriteLine("Timeout occurred");
}
else
{
Console.WriteLine(ex);
}
}
finally
{
if (resp != null)
{
resp.Close();
}
((ManualResetEvent)_state).Set();
}
}
上述方法通过ThreadPool.QueueUserWorkItem(waitCallback，stateObject)排队. ManualResetEvent用于控制排队行为，以使整个线程池不会被等待的任务填满(因为HttpWebRequest隐式使用工作线程，因为HttpWebRequest在内部实现异步功能以实现超时功能).
The above method is queued via ThreadPool.QueueUserWorkItem(waitCallback, stateObject). The ManualResetEvent is used to control queuing behavior so that not the entire thread pool gets filled up with waiting tasks (since the HttpWebRequest implicitly uses worker threads because it functions asynchronously internally to implement the timeout functionality).
所有这些的问题是，一旦HttpWebRequest的基础ServicePoint的所有连接都被使用"(即被远程服务器关闭)，就不会再打开新连接了. ServicePoint的ConnectionLeaseTimeout是否设置为较低的值(10秒)也没有关系.一旦系统进入此状态，它将不会再正常运行，因为它不会自动重新连接，并且所有后续的HttpWebRequests都将超时.现在的问题确实是，是否有办法通过某种方式在特定条件下销毁ServicePoint或关闭基础连接来解决此问题(我对ServicePoint.CloseConnectionGroup()并没有任何运气，但是该方法在如何使用方面还没有记载)正确使用它.)
The problem with all this is that once all connections of the HttpWebRequest's underlying ServicePoint are "used up" (i.e. closed by the remote server), there will be no new ones opened up. It also does not matter if the ConnectionLeaseTimeout of the ServicePoint is set to a low value (10 seconds). Once the system gets into this state, it will no longer function properly because it does not reconnect automatically and all subsequent HttpWebRequests will time out. Now the question really is if there is a way to solve this by somehow destroying a ServicePoint under certain conditions or closing underlying connections (I did not have any luck with ServicePoint.CloseConnectionGroup() yet, the method is also pretty undocumented in terms of how to properly use it).
有人知道我如何解决这个问题吗?
Does anybody have any idea how I could approach this problem?
推荐答案
基于这里的一些想法，我想到的解决方案是自己管理连接.如果将唯一的ConnectionGroupName分配给WebRequest(例如Guid.NewGuid().ToString())，则将在ServicePoint中为该请求创建一个具有一个连接的新连接组.请注意，此时没有更多的连接限制，因为.NET限制了每个连接组而不是每个ServicePoint，因此您必须自己处理.您将要重用连接组，以便重用与KeepAlive的现有连接，但是如果发生WebException异常，则请求的连接组应该被销毁，因为它可能是过时的.这样的事情(为每个主机名创建一个新实例):
The solution I came up with based on some of the ideas here is to manage the connections myself. If a unique ConnectionGroupName is assigned to a WebRequest (e.g. Guid.NewGuid().ToString()), a new connection group with one connection will be created in the ServicePoint for the request. Note that there's no more connection limiting at this point, since .NET limits per connection group rather than per ServicePoint, so you'll have to handle it yourself. You'll want to reuse connection groups so that existing connections with KeepAlive are reused, but if a WebException exception occurs, the request's connection group should be destroyed since it might be stale. Something like this (create a new instance for each host name):
public class ConnectionManager {
private const int _maxConnections = 4;
private Semaphore _semaphore = new Semaphore(_maxConnections, _maxConnections);
private Stack<string> _groupNames = new Stack<string>();
public string ObtainConnectionGroupName() {
_semaphore.WaitOne();
return GetConnectionGroupName();
}
public void ReleaseConnectionGroupName(string name) {
lock (_groupNames) {
_groupNames.Push(name);
}
_semaphore.Release();
}
public string SwapForFreshConnection(string name, Uri uri) {
ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
servicePoint.CloseConnectionGroup(name);
return GetConnectionGroupName();
}
private string GetConnectionGroupName() {
lock (_groupNames) {
return _groupNames.Count != 0 ? _groupNames.Pop() : Guid.NewGuid().ToString();
}
}
}
这篇关于HttpWebRequest如何处理(过早)关闭基础TCP连接?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
