未完成的对象耗尽了内存 [英] Unfinalized objects exhausting memory

问题描述

我们在Tomcat上以约20个实例的阵列在AWS上的Tomcat中运行基于Jersey(1.x)的服务:实例变质":在大约4个小时的过程中，其堆和CPU使用率增加，直到堆已用完，CPU被固定.到那时，它会自动从负载均衡器中删除并最终被杀死.

We're running a Jersey (1.x) based service in Tomcat on AWS in an array of ~20 instances Periodically an instance "goes bad": over the course of about 4 hours its heap and CPU usage increase until the heap is exhausted and the CPU is pinned. At that point it gets automatically removed from the load balancer and eventually killed.

从这些实例检查堆转储，约有95％的内存已被java.lang.ref.Finalizer实例占用，该实例保存着各种东西，但大多数或全部与HTTPS相关连接sun.net.www.protocol.https.HttpsURLConnectionImpl，sun.security.ssl.SSLSocketImpl，各种加密对象).这些是我们使用Jersey的客户端库与外部Web服务建立的连接.来自正常"实例的堆转储并不表示任何问题.

Examining heap dumps from these instances, ~95% of the memory has been used up by an instance of java.lang.ref.Finalizer which is holding onto all sorts of stuff, but most or all of it is related to HTTPS connections sun.net.www.protocol.https.HttpsURLConnectionImpl, sun.security.ssl.SSLSocketImpl, various crypto objects). These are connections that we're making to an external webservice using Jersey's client library. A heap dump from a "healthy" instance doesn't indicate any sort of issue.

在负载相对较低的情况下，实例可以运行数天或数周而不会出现问题.随着负载的增加，实例故障的频率也随之增加(每天平均发生故障的时间约为40％).

Under relatively low load instances run for days or weeks without issue. As load increases, so does the frequency of instance failure (several per day by the time average CPU gets to ~40%).

我们的JVM args是:

Our JVM args are:

-XX:+UseG1GC -XX:MaxPermSize=256m -Xmx1024m -Xms1024m

我正在为垃圾收集指标添加JMX日志记录，但是我不确定我应该寻找什么.目前，我主要是在寻找可能引发此类失败或其他调查目标的想法.

I'm in the process of adding JMX logging for garbage collection metrics, but I'm not entirely clear what I should be looking for. At this point I'm primarily looking for ideas of what could kick off this sort of failure or additional targets for investigation.

推荐答案

这最终是由JVM错误引起的(不幸的是，我丢失了与我们所跟踪的特定链接的链接).升级到OpenJDK的较新版本(我们最终使用OpenJDK 1.7.0_50)解决了该问题，而无需对代码进行任何更改.

This ultimately turned out to be caused by a JVM bug (unfortunately I've lost the link to the specific one we tracked it down to). Upgrading to a newer version of OpenJDK (we ended up with OpenJDK 1.7.0_50) solved the issue without us making any changes to our code.

这篇关于未完成的对象耗尽了内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！