异步套接字IO导致堆碎片（和过多的内存使用） [英] Asynchronous Socket IO causing heap fragmentation (and excessive memory use)

问题描述

我们有一个用VB.Net编写的Windows服务，目前以4.0 Framework为目标。服务器侦听来自客户端的连接，并在收到客户端时创建一个自定义对象，并将套接字作为成员，然后继续侦听其他客户端。
每个连接对象都异步接收数据（Socket.BeginRecieve），并有一个异步发送数据的Send方法。服务进程在不同的线程池线程上执行的每个命令，如果有输出要发送到客户端，
它会使用输出调用该客户端的连接对象的SendBytes方法。

We have a Windows service written in VB.Net currently targeting 4.0 Framework. The server listens for connections from clients and, on receipt of one, creates a custom object with the socket as a member and then continues listening for additional clients. Each connection object receives data asynchronously (Socket.BeginRecieve) and has a Send method that sends data asynchronously. Each command that the service processes executes on a different threadpool thread and, if there's output to be sent to the client, it calls that client's connection object's SendBytes method with the output.

Protected Friend Sub SendBytes(ByVal aData As Byte())
   If m_clientSocket Is Nothing _
         OrElse Not m_clientSocket.Connected _
         OrElse Not m_clientSocket.IsBound Then
      StopConn(True)
      Return
   End If
   If aData Is Nothing OrElse aData.Length < 1 Then Return
  Try
   m_clientSocket.BeginSend(aData, 0, aData.Length, SocketFlags.None, New AsyncCallback(AddressOf Me.Sent), m_clientSocket)
   Return
  Catch exSck As System.Net.Sockets.SocketException
   If exSck.ErrorCode = 10054 Then
    System.Diagnostics.Debug.WriteLine("Client connection was lost")
   End If
  Catch ex As Exception
   ' Log error
  End Try
  StopConn(True)
  Return
End Sub

Private Sub Sent(ByVal ar As IAsyncResult)
   Try
      Dim sentSocket As Socket = CType(ar.AsyncState, Socket)
      sentSocket.EndSend(ar)
   Catch ex As ArgumentNullException
      ' Log error
   Catch ex As ArgumentException
      ' Log error
   Catch ex As ObjectDisposedException
      ' Log error
      StopConn(True)
   Catch ex As InvalidOperationException
      ' Log error
   Catch ex As SocketException
      ' Log error
      StopConn(True)
   End Try
End Sub

StopConn是基类中的一个方法，用于从集合中删除连接对象，并且：

StopConn is a method in the base class that removes the connection object from the collection and:

If m_clientSocket IsNot Nothing Then
   '** Shutdown the socket
   m_clientSocket.Shutdown(SocketShutdown.Both)
   m_clientSocket.Close()
   m_clientSocket = Nothing
End If

然后处理连接对象。

随着时间的推移，服务的内存使用率会上升（不稳定，但逐渐增加），直到它最终停止工作并且必须重新启动。在最近的内存转储中，！gchandles显示了近200,000个异步固定句柄和相同数量的System.Threading.OverlappedData
对象。

then disposes the connection object.

Over time, the memory usage of the service climbs (not steadily, but gradually) until it eventually stops working and has to be restarted. In the most recent memory dump, !gchandles shows almost 200,000 Async Pinned Handles and the same number of System.Threading.OverlappedData objects.

堆的转储显示大自由段的数量，以及转储这些空闲段周围的内容会显示经常发送给客户端的更新消息。

A dump of the heap shows a large number of Free segments, and dumping the contents around those free segments reveals update messages that are frequently sent to the clients.

问题似乎与此知识库文章中描述的完全相同：

http://support.microsoft.com/kb/947862

The problem seems to be exactly what is described in this knowledge base article:
http://support.microsoft.com/kb/947862

本文的解决方案是：

"网络应用程序应该对其发布的*未完成*异步IO的数量设置上限。"

The solution from the article is:
"The network application should have an upper bound on the number of *outstanding* asynchronous IO that it posts."

我从这篇文章中得到的是，我有太多同时发送的"发送"信息。 （并接收？） 发生并需要限制（排队？）它们。我已经阅读了几篇关于这篇文章的论坛帖子，都在询问如何*最好*完成这个以及如何
确定上限。在论坛帖子中，人们普遍认为上限是机器依赖的，因为涉及的因素很多，但似乎没有人对如何确定它（甚至做出有根据的猜测）有答案，
或如何修复应用程序，以免发生这种情况。

What I'm getting from this article is that I have too many simultaneous "sends" (and receives?) happening and need to limit (queue?) them. I've read a couple of forum posts regarding this article, all asking how *best* to accomplish this and how to determine the upper bound. In the forum posts, there was general consensus that the upper bound would be machine dependent, because there were so many factors involved, but nobody seemed to have an answer for how to determine it (or even make an educated guess), or how to fix the application so this doesn't happen.

任何帮助或指示都将不胜感激。在此先感谢。

Any help or pointers would be greatly appreciated. Thanks in advance.

推荐答案

我不认为这篇文章直接适用于您的案例，因为您得到了一个 记忆力增长缓慢。 该文章引用了大量客户端同时连接到服务器。 结果可能类似于你有一个内存
泄漏但在你的情况下它可能是可解决的。 我考虑在"套接字数"上加上一个上限。一个绷带防止服务器崩溃，但它没有解决问题的原因。

I don't think the article directly applies in your case because you get a  slow growth of memory.  the article is reffereing to large number of clients simulataneously connecting to a server.  The results may be similar that you have a memory leak but in your case it may be solvable.  I consider putting an upper bound on the "number of sockets" a bandage in preventing the server from crashing but it isn't solving the roo cause of the problem.

我想知道你是否正在记录异常以及在内存不足之前获得的类型异常。 我怀疑插座不正确地关闭了。 当数据结束时，只有连接的一端应该启动
FIN命令。  Iff两端连接尝试关闭连接问题都可能发生。 如果这是你的问题，那就不一定了。

I would like to know if you are getting exceptions being logged and what type exceptions you are getting before you run out of memory.  I suspect that the sockets are getting closed improperly.  Only one end of a connection should initiate the FIN comand when the end of the data is reached.  Iff both ends of the connection attempt to close a connection problems can occur.  Not surre if this is the problem in your case.

我会使用DOS命令"Netstat -a"。获取PC上打开端口数的状态，并查看每个端口的状态。 我想你会发现端口没有正常关闭。 使用wireshark进行跟踪可以
有时会提供问题原因的线索。

I would use the DOS command "Netstat -a" to get a status of the number of open ports on your PC and see what the state of the each of tghe port are.  I think you will find the the ports aren't closing properly.  Getting a trace with wireshark can sometimes give clues to the cause of the problem.

这篇关于异步套接字IO导致堆碎片（和过多的内存使用）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！