长期存在的Java WeakReferences [英] Long lived Java WeakReferences

问题描述

我目前正在尝试诊断应用程序中的缓慢内存泄漏。我到目前为止的事实如下。

I am currently trying to diagnose a slow memory leak in my application. The facts I have so far are as follows.

• 我从应用程序的4天运行中获得了堆转储。


• 这个堆转储包含~800个WeakReference对象，这些对象指向对象（所有相同的类型，我将这个问题称为Foo）保留40mb的内存。


• Eclipse内存分析工具显示这些WeakReferences引用的每个Foo对象都不会被任何其他对象引用。我的期望是这应该使这些Foo对象弱可达，因此它们应该在下一个GC收集。


• 这些Foo对象中的每一个都有一个时间戳，表明它们是在课程中分配的4天的运行。在这段时间里我也有日志确认垃圾收集正在发生。


• 我的应用程序正在创建大量的Foo对象，其中只有很小一部分最终在堆转储中的这个状态。这告诉我，根本原因是某种竞争条件。


• 我的应用程序使用JNI来调用本机库。 JNI代码在初始化初始化期间调用NewGlobalRef 4次以获取对它使用的Java类的引用。

• I have a heap dump from a 4 day run of the application.
• This heap dump contains ~800 WeakReference objects which point to objects (all of the same type, which I will call Foo for the purposes of this question) retaining 40mb of memory.
• Eclipse Memory Analysis Tool shows that each of the Foo objects referred to by these WeakReferences is not referred to by any other objects. My expectation is that this should make these Foo objects Weakly Reachable and thus they should be collected at the next GC.
• Each of these Foo objects has a timestamp which shows that they were allocated over the course of the 4 day run. I also have logs during this time which confirm that Garbage Collection was happening.
• A huge number of Foo objects are being created by my application and only a very small fraction of them are ending up in this state within the heap dump. This suggests to me that the root cause is some sort of race condition.
• My application uses JNI to call through to a native library. The JNI code calls NewGlobalRef 4 times during start of day initialisation to get references to Java classes which it uses.

可能导致这些的原因尽管只是被WeakReferences引用（根据Eclipse Memory Analyzer工具），但Foo类仍未被收集？

What could possibly cause these Foo classes to not be collected despite only being referenced by WeakReferences (according to Eclipse Memory Analyser Tool)?

EDIT1：

@mindas
我使用的WeakReference等效于以下示例代码。

@mindas The WeakReference I am using is equivalent to the following example code.

public class FooWeakRef extends WeakReference<Foo>
{
  public long longA;
  public long longB;
  public String stringA;

  public FooWeakRef(Foo xiObject, ReferenceQueue<Foo> xiQueue)
  {
    super(xiObject, xiQueue);
  }
}

Foo没有终结器，任何终结器都不会只要WeakRefs尚未被清除就是一个考虑因素。当一个对象弱到达时，它不能最终确定。 有关详细信息，请参阅此页。

Foo does not have a finalizer and any finalizer would not be a consideration so long as the WeakRefs have not been cleared. An object is not finalizable when it is weakly reachable. See this page for details.

@kasten在对象可最终化之前清除弱引用。我的堆转储显示没有发生这种情况。

@kasten The weakreferences are cleared before the object is finalizable. My heap dump shows that this has not happened.

@jarnbjo我引用WeakReference Javadoc：

@jarnbjo I refer to the WeakReference Javadoc:

假设垃圾收集器在某个时间点确定一个对象是弱可达的。那时它将原子地清除对该对象的所有弱引用以及对该对象可通过的任何其他弱可达对象的所有弱引用。一系列强大而柔和的引用。

"Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references."

这表明GC应该检测到我的Foo对象是弱可达和当时清除弱参考。

This suggests to me that the GC should be detecting the fact that my Foo objects are "Weakly reachable" and "At that time" clearing the weak references.

编辑2

@j flemm - 我知道40mb不响同样多，但我担心4天内40mb意味着100天内4000mb。我读过的所有文档都表明，弱可达的对象不应该闲置几天。因此，我对如何在没有引用显示在堆转储中强烈引用对象的任何其他解释感兴趣。

@j flemm - I know that 40mb doesn't sound like much but I am worried that 40mb in 4 days means 4000mb in 100 days. All of the docs I have read suggest that objects which are weakly reachable should not hang around for several days. I am therefore interested in any other explanations about how an object could be strongly referenced without the reference showing up in a heap dump.

我将尝试分配一些大对象当其中一些悬空的Foo对象存在时，看看JVM是否收集它们。但是，此测试需要几天时间才能完成设置。

I am going to try allocating some large objects when some of these dangling Foo objects are present and see whether the JVM collects them. However, this test will take a couple of days to setup and complete.

编辑3

@jarnbjo - 我知道我不能保证JDK何时会注意到一个对象是弱可达的。但是，我希望在4天的重负载下应用程序可以为GC提供足够的机会来注意我的对象是弱可达的。 4天后，我非常怀疑其余的弱引用对象是否已经泄露。

@jarnbjo - I understand that I have no guarantee about when the JDK will notice that an object is weakly reachable. However, I would expect that an application under heavy load for 4 days would provide enough opportunities for the GC to notice that my objects are weakly reachable. After 4 days I am strongly suspicious that the remaining weakly references objects have been leaked somehow.

编辑4

@j flemm - 这真的很有趣！只是为了澄清，你是说GC正在你的应用程序上发生并且没有清除Soft / Weak refs？您能否告诉我有关您正在使用的JVM + GC配置的更多详细信息？我的应用程序使用80％堆的内存条来触发GC。我假设任何旧的GC的GC都会清除Weak refs。您是否建议GC仅在内存使用率高于较高阈值时收集弱引用？这个上限是否可配置？

@j flemm - Thats really interesting! Just to clarify, are you saying that GC is happening on your app and is not clearing Soft/Weak refs? Can you give me any more details about what JVM + GC Config you are using? My app is using a memory bar at 80% of the heap to trigger GC. I was assuming that any GC of the old gen would clear Weak refs. Are you suggesting that a GC only collects Weak refs once the memory usage is above a higher threshold? Is this higher limit configurable?

编辑5

@j flemm - 关于在SoftRefs之前清除WeakRefs的评论与Javadoc一致表明：
SoftRef：假设垃圾收集器在某个时间点确定一个对象可以轻柔地到达。那时它可能选择自动清除对该对象的所有软引用以及对任何其他可通过一系列强引用访问该对象的软可访问对象的所有软引用。同时或稍后它会将那些新清除的软引用排入队列。注册参考队列。

@j flemm - Your comment about clearing out WeakRefs before SoftRefs is consistent with the Javadoc which states: SoftRef: "Suppose that the garbage collector determines at a certain point in time that an object is softly reachable. At that time it may choose to clear atomically all soft references to that object and all soft references to any other softly-reachable objects from which that object is reachable through a chain of strong references. At the same time or at some later time it will enqueue those newly-cleared soft references that are registered with reference queues."

WeakRef：假设垃圾收集器在某个时间点确定一个对象是弱可达的。那时它将原子清除对该对象的所有弱引用以及对该对象所从的任何其他弱可达对象的所有弱引用通过一系列强大而柔软的参考资料获得。同时，它将声明所有以前弱可达的对象可以最终确定。在同一时间或稍后，它会将那些在参考队列中注册的新清除的弱引用排入队列。

WeakRef: "Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable. At that time it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. At the same time it will declare all of the formerly weakly-reachable objects to be finalizable. At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues."

为清楚起见，你是说垃圾了吗？当你的应用程序有超过50％的可用内存时收集器运行，在这种情况下它不能清除WeakRefs？当你的应用程序有> 50％的可用内存时，为什么GC会运行？我认为你的应用程序可能只是产生了一个非常低的垃圾量和收集器运行时清除WeakRef但不清除SoftRef。

For clarity, are you saying that the Garbage Collector runs when your app has more than 50% free memory and in this case it does not clear WeakRefs? Why would the GC run at all when your app has >50% free memory? I think your app is probably just generating a very low amount of garbage and when the collector runs it is clearing WeakRefs but not SoftRefs.

编辑6

@j flemm - 对你的应用程序行为的另一个可能的解释是年轻人正在收集，但你的弱和软件都是旧的，并且只有在收集旧版本时才会被清除。对于我的应用程序，我有统计数据显示正在收集旧版本，这应该意味着WeakRefs被清除。

@j flemm - The other possible explanation for your app's behaviour is that the young gen is being collected but that your Weak and Soft refs are all in the old gen and are only cleared when the old gen is being collected. For my app I have stats showing that the old gen is being collected which should mean that WeakRefs get cleared.

编辑7

我正在开始关于这个问题的赏金。我正在寻找任何一个问题关于如何在GC发生时无法清除WeakRefs的解释。如果答案是这是不可能的，那么理想情况下我会指向OpenJDK的相应位，它表示一旦确定一个对象是弱可达的，就会清除WeakRefs，并且每次GC运行时都会解析弱可达性。

I am starting a bounty on this question. I am looking for any plausible explanations for how WeakRefs could fail to be cleared while GC is happening. If the answer is that this is impossible I would ideally like to be pointed at the appropriate bits of OpenJDK which show WeakRefs being cleared as soon as an object is determined to be weakly reachable and that weak reachability is resolved every time GC runs.

推荐答案

我终于检查了Hotspot JVM源代码并找到了以下代码。

I have finally got round to checking the Hotspot JVM source code and found the following code.

在referenceProcessor.cpp中：

In referenceProcessor.cpp:

void ReferenceProcessor::process_discovered_references(
  BoolObjectClosure*           is_alive,
  OopClosure*                  keep_alive,
  VoidClosure*                 complete_gc,
  AbstractRefProcTaskExecutor* task_executor) {
  NOT_PRODUCT(verify_ok_to_handle_reflists());

  assert(!enqueuing_is_done(), "If here enqueuing should not be complete");
  // Stop treating discovered references specially.
  disable_discovery();

  bool trace_time = PrintGCDetails && PrintReferenceGC;
  // Soft references
  {
    TraceTime tt("SoftReference", trace_time, false, gclog_or_tty);
    process_discovered_reflist(_discoveredSoftRefs, _current_soft_ref_policy, true,
                               is_alive, keep_alive, complete_gc, task_executor);
  }

  update_soft_ref_master_clock();

  // Weak references
  {
    TraceTime tt("WeakReference", trace_time, false, gclog_or_tty);
    process_discovered_reflist(_discoveredWeakRefs, NULL, true,
                               is_alive, keep_alive, complete_gc, task_executor);
  }

函数process_discovered_reflist具有以下签名：

The function process_discovered_reflist has the following signature:

void
ReferenceProcessor::process_discovered_reflist(
  DiscoveredList               refs_lists[],
  ReferencePolicy*             policy,
  bool                         clear_referent,
  BoolObjectClosure*           is_alive,
  OopClosure*                  keep_alive,
  VoidClosure*                 complete_gc,
  AbstractRefProcTaskExecutor* task_executor)

这表明WeakRefs被ReferenceProcessor :: process_discovered_references无条件清除。

This shows that WeakRefs are being unconditionally cleared by ReferenceProcessor::process_discovered_references.

搜索process_discovered_reference的热点代码显示CMS collector（我正在使用的）从以下调用堆栈调用此方法。

Searching the Hotspot code for process_discovered_reference shows that the CMS collector (which is what I am using) calls this method from the following call stack.

CMSCollector::refProcessingWork
CMSCollector::checkpointRootsFinalWork
CMSCollector::checkpointRootsFinal

这个调用堆栈看起来好像每次运行CMS集合时都会被调用。

This call stack looks like it is invoked every time a CMS collection is run.

假设这是真的，对长期存在的弱引用对象的唯一解释是微妙的JVM错误或者如果GC没有运行。

Assuming this is true, the only explanation for a long lived weakly referenced object would be either a subtle JVM bug or if the GC had not been run.

这篇关于长期存在的Java WeakReferences的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！