如何编写可扩展的基于 TCP/IP 的服务器 [英] How to write a scalable TCP/IP based server
问题描述
我正处于编写一个新的 Windows 服务应用程序的设计阶段，该应用程序接受 TCP/IP 连接以进行长时间运行的连接(即，这不像 HTTP 那样有许多短连接，而是客户端连接并保持连接数小时或数天甚至数周).
I am in the design phase of writing a new Windows service application that accepts TCP/IP connections for long running connections (i.e., this is not like HTTP where there are many short connections, but rather a client connects and stays connected for hours or days or even weeks).
我正在寻找设计网络架构的最佳方式的想法.我将需要为该服务至少启动一个线程.我正在考虑使用异步 API(BeginRecieve 等)，因为我不知道在任何给定时间(可能是数百个)我将连接多少个客户端.我绝对不想为每个连接都启动一个线程.
I'm looking for ideas for the best way to design the network architecture. I'm going to need to start at least one thread for the service. I am considering using the Asynch API (BeginRecieve, etc.) since I don't know how many clients I will have connected at any given time (possibly hundreds). I definitely do not want to start a thread for each connection.
数据将主要从我的服务器流出到客户端，但有时会从客户端发送一些命令.这主要是一个监控应用程序，我的服务器定期向客户端发送状态数据.
Data will primarily flow out to the clients from my server, but there will be some commands sent from the clients on occasion. This is primarily a monitoring application in which my server sends status data periodically to the clients.
使它尽可能具有可扩展性的最佳方法是什么?基本工作流程?
What is the best way to make this as scalable as possible? Basic workflow?
明确地说，我正在寻找基于 .NET 的解决方案(如果可能，C#，但任何 .NET 语言都可以).
To be clear, I'm looking for .NET-based solutions (C# if possible, but any .NET language will work).
我需要一个解决方案的工作示例，作为指向我可以下载的内容的指针或内嵌的简短示例.并且它必须是基于 .NET 和 Windows 的(任何 .NET 语言都可以接受).
I would need a working example of a solution, either as a pointer to something I could download or a short example in-line. And it must be .NET and Windows based (any .NET language is acceptable).
推荐答案
我以前写过类似的东西.我多年前的研究表明，使用异步套接字编写自己的套接字实现是最好的选择.这意味着没有真正做任何事情的客户端实际上需要相对较少的资源.发生的任何事情都由 .NET 线程池处理.
I've written something similar to this in the past. From my research years ago showed that writing your own socket implementation was the best bet, using the asynchronous sockets. This meant that clients not really doing anything actually required relatively few resources. Anything that does occur is handled by the .NET thread pool.
我把它写成一个管理服务器所有连接的类.
I wrote it as a class that manages all connections for the servers.
我只是使用了一个列表来保存所有客户端连接，但是如果您需要更快地查找更大的列表，您可以随意编写它.
I simply used a list to hold all the client connections, but if you need faster lookups for larger lists, you can write it however you want.
private List<xConnection> _sockets;
此外，您还需要套接字实际侦听传入连接.
Also you need the socket actually listening for incoming connections.
private System.Net.Sockets.Socket _serverSocket;
start 方法实际上启动了服务器套接字并开始侦听任何传入的连接.
The start method actually starts the server socket and begins listening for any incoming connections.
public bool Start()
{
System.Net.IPHostEntry localhost = System.Net.Dns.GetHostEntry(System.Net.Dns.GetHostName());
System.Net.IPEndPoint serverEndPoint;
try
{
serverEndPoint = new System.Net.IPEndPoint(localhost.AddressList[0], _port);
}
catch (System.ArgumentOutOfRangeException e)
{
throw new ArgumentOutOfRangeException("Port number entered would seem to be invalid, should be between 1024 and 65000", e);
}
try
{
_serverSocket = new System.Net.Sockets.Socket(serverEndPoint.Address.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
}
catch (System.Net.Sockets.SocketException e)
{
throw new ApplicationException("Could not create socket, check to make sure not duplicating port", e);
}
try
{
_serverSocket.Bind(serverEndPoint);
_serverSocket.Listen(_backlog);
}
catch (Exception e)
{
throw new ApplicationException("An error occurred while binding socket. Check inner exception", e);
}
try
{
//warning, only call this once, this is a bug in .net 2.0 that breaks if
// you're running multiple asynch accepts, this bug may be fixed, but
// it was a major pain in the rear previously, so make sure there is only one
//BeginAccept running
_serverSocket.BeginAccept(new AsyncCallback(acceptCallback), _serverSocket);
}
catch (Exception e)
{
throw new ApplicationException("An error occurred starting listeners. Check inner exception", e);
}
return true;
}
我只想指出异常处理代码看起来很糟糕，但原因是我在那里有异常抑制代码，以便任何异常都会被抑制并返回 false 如果配置选项已设置，但为了简洁起见，我想将其删除.
I'd just like to note the exception handling code looks bad, but the reason for it is I had exception suppression code in there so that any exceptions would be suppressed and return false if a configuration option was set, but I wanted to remove it for brevity sake.
上面的 _serverSocket.BeginAccept(new AsyncCallback(acceptCallback)), _serverSocket) 实质上设置了我们的服务器套接字，以便在用户连接时调用 acceptCallback 方法.此方法从 .NET 线程池运行，如果您有许多阻塞操作，它会自动处理创建额外的工作线程.这应该可以最佳地处理服务器上的任何负载.
The _serverSocket.BeginAccept(new AsyncCallback(acceptCallback)), _serverSocket) above essentially sets our server socket to call the acceptCallback method whenever a user connects. This method runs from the .NET threadpool, which automatically handles creating additional worker threads if you have many blocking operations. This should optimally handle any load on the server.
private void acceptCallback(IAsyncResult result)
{
xConnection conn = new xConnection();
try
{
//Finish accepting the connection
System.Net.Sockets.Socket s = (System.Net.Sockets.Socket)result.AsyncState;
conn = new xConnection();
conn.socket = s.EndAccept(result);
conn.buffer = new byte[_bufferSize];
lock (_sockets)
{
_sockets.Add(conn);
}
//Queue receiving of data from the connection
conn.socket.BeginReceive(conn.buffer, 0, conn.buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), conn);
//Queue the accept of the next incoming connection
_serverSocket.BeginAccept(new AsyncCallback(acceptCallback), _serverSocket);
}
catch (SocketException e)
{
if (conn.socket != null)
{
conn.socket.Close();
lock (_sockets)
{
_sockets.Remove(conn);
}
}
//Queue the next accept, think this should be here, stop attacks based on killing the waiting listeners
_serverSocket.BeginAccept(new AsyncCallback(acceptCallback), _serverSocket);
}
catch (Exception e)
{
if (conn.socket != null)
{
conn.socket.Close();
lock (_sockets)
{
_sockets.Remove(conn);
}
}
//Queue the next accept, think this should be here, stop attacks based on killing the waiting listeners
_serverSocket.BeginAccept(new AsyncCallback(acceptCallback), _serverSocket);
}
}
上面的代码基本上刚刚接受了进来的连接，将BeginReceive排队，这是一个在客户端发送数据时会运行的回调，然后将下一个acceptCallback排队> 它将接受进入的下一个客户端连接.
The above code essentially just finished accepting the connection that comes in, queues BeginReceive which is a callback that will run when the client sends data, and then queues the next acceptCallback which will accept the next client connection that comes in.
BeginReceive 方法调用告诉套接字从客户端接收数据时要做什么.对于BeginReceive，你需要给它一个字节数组，这是客户端发送数据时复制数据的地方.ReceiveCallback 方法将被调用，这就是我们处理接收数据的方式.
The BeginReceive method call is what tells the socket what to do when it receives data from the client. For BeginReceive, you need to give it a byte array, which is where it will copy the data when the client sends data. The ReceiveCallback method will get called, which is how we handle receiving data.
private void ReceiveCallback(IAsyncResult result)
{
//get our connection from the callback
xConnection conn = (xConnection)result.AsyncState;
//catch any errors, we'd better not have any
try
{
//Grab our buffer and count the number of bytes receives
int bytesRead = conn.socket.EndReceive(result);
//make sure we've read something, if we haven't it supposadly means that the client disconnected
if (bytesRead > 0)
{
//put whatever you want to do when you receive data here
//Queue the next receive
conn.socket.BeginReceive(conn.buffer, 0, conn.buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), conn);
}
else
{
//Callback run but no data, close the connection
//supposadly means a disconnect
//and we still have to close the socket, even though we throw the event later
conn.socket.Close();
lock (_sockets)
{
_sockets.Remove(conn);
}
}
}
catch (SocketException e)
{
//Something went terribly wrong
//which shouldn't have happened
if (conn.socket != null)
{
conn.socket.Close();
lock (_sockets)
{
_sockets.Remove(conn);
}
}
}
}
在这种模式中，我忘了在这方面的代码中提到这一点:
In this pattern I forgot to mention that in this area of code:
//put whatever you want to do when you receive data here
//Queue the next receive
conn.socket.BeginReceive(conn.buffer, 0, conn.buffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), conn);
通常，在任何你想要的代码中，我都会将数据包重新组装成消息，然后将它们创建为线程池上的作业.这样，无论消息处理代码正在运行，客户端下一个块的 BeginReceive 都不会延迟.
Generally, in the whatever you want code, I would do reassembly of packets into messages, and then create them as jobs on the thread pool. This way the BeginReceive of the next block from the client isn't delayed while whatever message processing code is running.
accept 回调通过调用 end receive 完成对数据套接字的读取.这会填充开始接收函数中提供的缓冲区.一旦你在我留下评论的地方做任何你想做的事情，我们就会调用下一个 BeginReceive 方法，如果客户端发送更多数据，它将再次运行回调.
The accept callback finishes reading the data socket by calling end receive. This fills the buffer provided in the begin receive function. Once you do whatever you want where I left the comment, we call the next BeginReceive method which will run the callback again if the client sends any more data.
现在是真正棘手的部分:当客户端发送数据时，您的接收回调可能只用部分消息调用.重新组装会变得非常复杂.我使用我自己的方法并创建了一种专有协议来做到这一点.我省略了，但如果你要求，我可以添加它.这个处理程序实际上是我写过的最复杂的一段代码.
Now here's the really tricky part: When the client sends data, your receive callback might only be called with part of the message. Reassembly can become very very complicated. I used my own method and created a sort of proprietary protocol to do this. I left it out, but if you request, I can add it in. This handler was actually the most complicated piece of code I had ever written.
public bool Send(byte[] message, xConnection conn)
{
if (conn != null && conn.socket.Connected)
{
lock (conn.socket)
{
//we use a blocking mode send, no async on the outgoing
//since this is primarily a multithreaded application, shouldn't cause problems to send in blocking mode
conn.socket.Send(bytes, bytes.Length, SocketFlags.None);
}
}
else
return false;
return true;
}
上面的send方法实际上使用了一个同步的Send调用.对我来说这很好，因为我的应用程序的消息大小和多线程性质.如果你想发送到每个客户端，你只需要循环遍历 _sockets 列表.
The above send method actually uses a synchronous Send call. For me that was fine due to the message sizes and the multithreaded nature of my application. If you want to send to every client, you simply need to loop through the _sockets List.
你在上面看到的 xConnection 类基本上是一个简单的套接字包装器，用于包含字节缓冲区，在我的实现中还有一些额外的东西.
The xConnection class you see referenced above is basically a simple wrapper for a socket to include the byte buffer, and in my implementation some extras.
public class xConnection : xBase
{
public byte[] buffer;
public System.Net.Sockets.Socket socket;
}
这里也供参考的是我包含的 using s，因为当它们不包含时我总是很生气.
Also for reference here are the usings I include since I always get annoyed when they aren't included.
using System.Net.Sockets;
希望对你有帮助.它可能不是最干净的代码，但它可以工作.代码也有一些细微差别，您应该对更改感到厌烦.一方面，在任何时候只调用一个 BeginAccept.曾经有一个非常烦人的 .NET 错误，那是几年前的事了，所以我不记得细节了.
I hope that's helpful. It may not be the cleanest code, but it works. There are also some nuances to the code which you should be weary about changing. For one, only have a single BeginAccept called at any one time. There used to be a very annoying .NET bug around this, which was years ago so I don't recall the details.
此外，在 ReceiveCallback 代码中，我们会在下一次接收排队之前处理从套接字接收到的任何内容.这意味着对于单个套接字，我们实际上只在任何时间点 ReceiveCallback 中一次，并且我们不需要使用线程同步.但是，如果您重新排序以在拉取数据后立即调用下一个接收，这可能会快一点，您需要确保正确同步线程.
Also, in the ReceiveCallback code, we process anything received from the socket before we queue the next receive. This means that for a single socket, we're only actually ever in ReceiveCallback once at any point in time, and we don't need to use thread synchronization. However, if you reorder this to call the next receive immediately after pulling the data, which might be a little faster, you will need to make sure you properly synchronize the threads.
此外，我砍掉了很多代码，但保留了正在发生的事情的本质.这应该是您设计的良好开端.如果您对此还有任何疑问，请发表评论.
Also, I hacked out a lot of my code, but left the essence of what's happening in place. This should be a good start for you're design. Leave a comment if you have any more questions around this.
这篇关于如何编写可扩展的基于 TCP/IP 的服务器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
