Java“for”语句实现可防止垃圾收集 [英] Java "for" statement implementation prevents garbage collecting
问题描述
UPD 21.11.2017：该错误已在JDK中修复，请参阅 Vicente Romero的评论
摘要：
如果 for 语句，如果用于任何 Iterable 实现，集合将保留在堆内存中，直到当前作用域（方法，语句体）结束，并且不会被垃圾收集，即使您没有任何其他对该集合的引用，并且应用程序需要分配一个新的内存。
$ b http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8175883
：
如果我有下一个代码，它分配一个la列表包含随机内容的rge字符串：
import java.util.ArrayList;
public class IteratorAndGc {
//字符串的数量和每个字符串的大小
static final int N = 7500;
public static void main（String [] args）{
System.gc（）;
gcInMethod（）;
System.gc（）;
showMemoryUsage（方法主体之后的GC）;
ArrayList< String> strings2 = generateLargeStringsArray（N）;
showMemoryUsage（方法之外的第三次分配总是成功）;
//主要可测试方法
public static void gcInMethod（）{
showMemoryUsage（在第一次分配内存之前）;
ArrayList< String> strings = generateLargeStringsArray（N）;
showMemoryUsage（第一次内存分配后）;
//这只是一个区别 - 在迭代器创建之后，内存将不会被收集，直到函数结束
for（String string：strings）;
showMemoryUsage（迭代后）;
strings = null; //抛弃对数组的引用
//一个说这不能保证垃圾回收，
// Oracle说：Java虚拟机已经尽最大努力从空间中回收空间所有丢弃的对象。
//但无论如何 - 程序行为在使用或不使用此行时都保持不变。你可以跳过它并测试。
System.gc（）;
showMemoryUsage（在方法体中强制GC之后）;
尝试{
System.out.println（尝试再次在方法体中分配内存：）;
ArrayList< String> strings2 = generateLargeStringsArray（N）;
showMemoryUsage（二级内存分配后）;
catch（OutOfMemoryError e）{
showMemoryUsage（!!!! Out of memory error !!!!）;
System.out.println（）;
}
}
//函数分配并返回对大量内存的引用
private static ArrayList< String> generateLargeStringsArray（int N）{
ArrayList< String> strings = new ArrayList<>（N）;
for（int i = 0; i
}
strings.add（sb.toString（））;
}
返回字符串;
}
//辅助方法显示当前内存状态
public static void showMemoryUsage（String action）{
free free = Runtime.getRuntime（）。freeMemory ）;
long total = Runtime.getRuntime（）。totalMemory（）;
long max = Runtime.getRuntime（）。maxMemory（）;
long used = total - free;
System.out.printf（\t％40s：％10dk of max％10dk％n，action，used / 1024，max / 1024）;
$ b 编译并使用有限内存，像这样（180mb）：
javac IteratorAndGc.java&& java -Xms180m -Xmx180m IteratorAndGc
在运行时，我有：
在第一次分配内存之前：1251k最大176640k
第一次内存分配后：131426k最大176640k
迭代后：131426k最大176640k
在方法体中强制GC：最大176640k的110682k（几乎没有收集）
试着再次在方法体内分配内存：
！ ！内存不足错误!!!!：168948k的最大176640k
GC后方法体：459k最大176640k（垃圾被收集！）
第三次分配方法总是成功的：117740k最大163840k
所以，在我的gcInMethod（）里面，我尝试分配列表，迭代它，放弃对列表的引用，（可选）强制垃圾收集并分配类似再次列出。但是由于内存不足，我无法分配第二个数组。
同时，在函数体外，我可以成功地强制垃圾回收（可选）并再次分配相同的数组大小！
为了避免在函数体内出现这个，只需删除/注释这一行即可：
for（String string：strings）; < - 这是邪恶!!!
然后输出如下所示：
在第一次分配内存之前：1251k最大176640k
第一次内存分配后：131409k最大176640k
迭代后：131409k最大176640k
在方法体中强制GC之后：最大值为176640k的497k（垃圾被收集！）
尝试再次在方法体中分配内存：
第二次内存分配后：115541k最大163840k
GC之后的方法体：493k的最大163840k（垃圾收集！）
第三次分配方法总是成功：121300k最大为163840k
因此，如果没有 for 迭代垃圾，字符串，并且第二次分配（在函数体内部）并且第三次分配（在方法外部）。
我的推测是：
Iterator iter = strings.iterator（）语法结构编译为 $ b ;
while（iter.hasNext（））{
iter.next（）
}
（我检查了这个反编译 javap -c IteratorAndGc.class ）
这个 iter 引用停留在范围内直到结束。您无权访问该引用来取消它，并且GC无法执行该集合。
也许这是正常行为（甚至可能指定为 javac ，但我还没有找到），但恕我直言，如果编译器创建了一些实例，它应该关心在使用后将它们从范围中丢弃。
这就是我期望对语句执行：
Iterator iter = strings.iterator（）;
while（iter.hasNext（））{
iter.next（）
}
iter = null; //< ---冲洗水！
使用的java编译器和运行时版本：
javac 1.8.0_111
java版本1.8.0_111
Java™SE运行时环境（build 1.8.0_111-b14）
Java HotSpot™64位服务器虚拟机（构建25.111-b14，混合模式）
注意： 这个问题不是关于编程风格，最佳实践，
惯例问题是关于Java
平台的效率问题。 System.gc （）行为（您可以从示例中删除所有
gc 调用） - 在第二个字符串分配期间，JVM 必须释放dicarded memory。
引用测试java类，在线编译器测试（但是这个资源只有50 Mb的堆，所以使用N = 5000）
最后，Oracle / Open JKD bug被接受，批准并修复：
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8175883
https://bugs.openjdk.java.net/browse/ JDK-8175883
引用线程的意见：
这是一个可以在8和9上重现的问题
有一些问题，程序保留它自己隐式自动生成的
引用内存块直到下一个隐式用法及其内存
被锁定导致OOM
（这证明 @ vanza的期望，参见来自JDK开发者的这个例子）
lockquote
根据规范，这不应该发生
（这是我的问题的答案：如果编译器创建了一些实例，它应该）））
UPD 21.11.2017：该错误已在JDK中修复，请参阅<来自Vicente Romero的评论
UPD 21.11.2017: the bug is fixed in JDK, see comment from Vicente Romero
Summary:
If for statement if used for any Iterable implementation the collection will remain in the heap memory till the end of current scope (method, statement body) and won't be garbage collected even if you don't have any other references to the collection and the application needs to allocate a new memory.
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8175883
https://bugs.openjdk.java.net/browse/JDK-8175883
The example:
If i have the next code, which allocates a list of large strings with random content:
import java.util.ArrayList;
public class IteratorAndGc {
// number of strings and the size of every string
static final int N = 7500;
public static void main(String[] args) {
System.gc();
gcInMethod();
System.gc();
showMemoryUsage("GC after the method body");
ArrayList<String> strings2 = generateLargeStringsArray(N);
showMemoryUsage("Third allocation outside the method is always successful");
}
// main testable method
public static void gcInMethod() {
showMemoryUsage("Before first memory allocating");
ArrayList<String> strings = generateLargeStringsArray(N);
showMemoryUsage("After first memory allocation");
// this is only one difference - after the iterator created, memory won't be collected till end of this function
for (String string : strings);
showMemoryUsage("After iteration");
strings = null; // discard the reference to the array
// one says this doesn't guarantee garbage collection,
// Oracle says "the Java Virtual Machine has made a best effort to reclaim space from all discarded objects".
// but no matter - the program behavior remains the same with or without this line. You may skip it and test.
System.gc();
showMemoryUsage("After force GC in the method body");
try {
System.out.println("Try to allocate memory in the method body again:");
ArrayList<String> strings2 = generateLargeStringsArray(N);
showMemoryUsage("After secondary memory allocation");
} catch (OutOfMemoryError e) {
showMemoryUsage("!!!! Out of memory error !!!!");
System.out.println();
}
}
// function to allocate and return a reference to a lot of memory
private static ArrayList<String> generateLargeStringsArray(int N) {
ArrayList<String> strings = new ArrayList<>(N);
for (int i = 0; i < N; i++) {
StringBuilder sb = new StringBuilder(N);
for (int j = 0; j < N; j++) {
sb.append((char)Math.round(Math.random() * 0xFFFF));
}
strings.add(sb.toString());
}
return strings;
}
// helper method to display current memory status
public static void showMemoryUsage(String action) {
long free = Runtime.getRuntime().freeMemory();
long total = Runtime.getRuntime().totalMemory();
long max = Runtime.getRuntime().maxMemory();
long used = total - free;
System.out.printf("\t%40s: %10dk of max %10dk%n", action, used / 1024, max / 1024);
}
}
compile and run it with limited memory, like this (180mb):
javac IteratorAndGc.java && java -Xms180m -Xmx180m IteratorAndGc
and at runtime i have:
Before first memory allocating: 1251k of max 176640k
After first memory allocation: 131426k of max 176640k
After iteration: 131426k of max 176640k
After force GC in the method body: 110682k of max 176640k (almost nothing collected)
Try to allocate memory in the method body again:
!!!! Out of memory error !!!!: 168948k of max 176640kGC after the method body: 459k of max 176640k (the garbage is collected!)
Third allocation outside the method is always successful: 117740k of max 163840k
So, inside gcInMethod() i tried to allocate the list, iterate over it, discard the reference to the list, (optional)force garbage collection and allocate similar list again. But i can't allocate second array because of lack of memory.
In the same time, outside the function body i can successfully force garbage collection (optional) and allocate the same array size again!
To avoid this OutOfMemoryError inside the function body it's enough to remove/comment only this one line:
for (String string : strings); <-- this is the evil!!!
and then output looks like this:
Before first memory allocating: 1251k of max 176640k
After first memory allocation: 131409k of max 176640k
After iteration: 131409k of max 176640k
After force GC in the method body: 497k of max 176640k (the garbage is collected!)
Try to allocate memory in the method body again:
After secondary memory allocation: 115541k of max 163840k
GC after the method body: 493k of max 163840k (the garbage is collected!)
Third allocation outside the method is always successful: 121300k of max 163840k
So, without for iterating the garbage successfully collected after discarding the reference to the strings, and allocated second time (inside the function body) and allocated third time (outside the method).
My supposition:
for syntax construction is compiled to
Iterator iter = strings.iterator();
while(iter.hasNext()){
iter.next()
}
(and i checked this decompiling javap -c IteratorAndGc.class)
And looks like this iter reference stays in the scope till the end. You don't have access to the reference to nullify it, and GC can't perform the collection.
Maybe this is normal behavior (maybe even specified in javac, but i haven't found), but IMHO if compiler creates some instances it should care about discarding them from the scope after using.
That's how i expect to have the implementation of for statement:
Iterator iter = strings.iterator();
while(iter.hasNext()){
iter.next()
}
iter = null; // <--- flush the water!
Used java compiler and runtime versions:
javac 1.8.0_111
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
Note:
the question is not about programming style, best practices, conventions and so on, the question is about an efficiency of Java platform.
the question is not about
System.gc()behavior (you may remove all gc calls from the example) - during the second strings allocation the JVM must release the dicarded memory.
Reference to the test java class, Online compiler to test (but this resource has only 50 Mb of heap, so use N = 5000)
Finally, Oracle/Open JKD bug is accepted, approved and fixed:
http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8175883
https://bugs.openjdk.java.net/browse/JDK-8175883
Quoting the comments from the threads:
This is an issue reproducible both on 8 and 9
There is some issue the program keeps it's own implicit auto-generated reference to the memory block till next implicit usage and its memory is being locked that causing OOM
(this proves @vanza's expectation, see this example from the JDK developer)
According the spec this should not happen
(this is an answer to my question: if compiler creates some instances it should care about discarding them from the scope after using)
UPD 21.11.2017: the bug is fixed in JDK, see comment from Vicente Romero
这篇关于Java“for”语句实现可防止垃圾收集的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
