JNA / ByteBuffer的没有得到释放，造成ç堆内存耗尽 [英] JNA/ByteBuffer not getting freed and causing C heap to run out of memory
问题描述
让我说我怎么JNA和Java本机直接内存分配的理解是内脏充其量开始，所以我试图来形容我这是怎么回事的理解。除了回应任何更正将是巨大的......
Let me start by saying that my understanding of how JNA and Java direct native memory allocations is visceral at best, so I'm trying to describe my understanding of what's going on. Any corrections in addition to responses would be great...
我正在使用JNA混合了Java和C本地code的应用程序和正在运行翻过与Java垃圾收集未能自由引用一个可重复的问题，直接本机内存分配，导致C堆中耗尽内存。
I'm running an application that mixes Java and C native code using JNA and am running accross a reproducible issue with the Java Garbage Collector failing to free references to direct native memory allocations, resulting in the C heap running out of memory.
我敢肯定，我的C应用程序不分配问题的根源，因为我传递一个 java.nio.ByteBuffer中进入我的C code，修改缓冲区，然后在访问我的Java功能的结果。我有一个的malloc 和一个对应的免费每个函数调用过程中，但经过反复运行code在Java中的malloc最终将失败。
I'm positive that my C application is not the source of the allocation issue, as I'm passing a java.nio.ByteBuffer into my C code, modifying the buffer, and then accessing the result in my Java function. I have a single malloc and a single corresponding free during each function call, but after repeatedly running the code in Java the malloc will eventually fail.
下面是呈现问题code的有点轻视集 - 实事求是我试图在函数调用的过程中C堆中分配约为16-32MB
Here's a somewhat trivialized set of code that exhibits the issue -- realistically I'm trying to allocate about 16-32MB on the C heap during the function call.
我的Java code的确是这样的：
My Java code does something like:
public class MyClass{
public void myfunction(){
ByteBuffer foo = ByteBuffer.allocateDirect(1000000);
MyDirectAccessLib.someOp(foo, 1000000);
System.out.println(foo.get(0));
}
}
public MyDirectAccessLib{
static {
Native.register("libsomelibrary");
}
public static native void someOp(ByteBuffer buf, int size);
}
然后我的C code可能是这样的：
Then my C code might be something like:
#include <stdio.h>
#include <stdlib.h>
void someOp(unsigned char* buf, int size){
unsigned char *foo;
foo = malloc(1000000);
if(!foo){
fprintf(stderr, "Failed to malloc 1000000 bytes of memory\n");
return;
}
free(foo);
buf[0] = 100;
}
问题是调用此函数后反复Java堆稍微稳定的（它生长缓慢），但C函数最终无法分配任何更多的内存。在较高的水平，我相信这是因为Java的内存分配到C堆，而不是清理指向这个内存，因为Java ByteBuffer的对象是比较小的ByteBuffer的。
Trouble is after calling this function repeatedly the Java heap is somewhat stable (it grows slowly), but the C function eventually cannot allocate any more memory. At a high level I believe this is because Java is allocating memory to the C heap, but not cleaning up the ByteBuffer that points at this memory because the Java ByteBuffer object is relatively small.
到目前为止，我发现在我的功能将提供所需清理手动运行GC，但是这似乎是既贫穷的想法和一个贫穷的解决方案。
Thus far I've found running the GC manually in my function will provide the required cleanup, but this seems like both a poor idea and a poor solution.
我如何管理这个问题更好地使ByteBuffer的空间适当释放，我的C堆空间被控制？
How can I manage this problem better so that the ByteBuffer space is appropriately freed and my C heap space is controlled?
是我的问题不正确的理解（是有什么我不当运行）？
Is my understanding of the problem incorrect (is there something I'm running improperly)?
修改：调整缓冲区大小，以更好地反映我的实际应用，我对分配大约图像... 3000x2000音乐
Edit: adjusted buffer sizes to be more reflective of my actual application, I'm allocating for images approximately 3000x2000...
推荐答案
我想你已经正确诊断：你永远不会耗尽Java堆，所以JVM不垃圾收集，并映射缓冲区并不释放。你没有运行GC的问题时，这一事实手工似乎证实了这一点。你也可以打开详细日志记录收集作为二次确认。
I think that you've diagnosed properly: you never run out of Java heap, so the JVM doesn't garbage collect, and the mapped buffers aren't freed. The fact that you don't have problems when running GC manually seems to confirm this. You could also turn on verbose collection logging as a secondary confirmation.
所以，你能做些什么？好吧，我想尝试的第一件事是要保持初始JVM堆体积小，使用-Xms命令行参数。这可能会导致问题，如果你的程序不断对Java堆分配少量的内存，因为它会GC运行更频繁。
So what can you do? Well, first thing I'd try is to keep the initial JVM heap size small, using the -Xms command-line argument. This can cause problems, if your program is constantly allocating small amounts memory on the Java heap, as it will run GC more frequently.
我还使用的 PMAP 的工具（或任何等值是在Windows上）检查虚拟内存映射。这有可能是你正在分裂C堆中，通过分配可变大小的缓冲区。如果是这样的话，那么你会看到每一个更大的虚拟地图，以匿名块之间的差距。而且该解决方案有分配恒定大小的块是比你需要更大的。
I'd also use the pmap tool (or whatever its equivalent is on Windows) to examine the virtual memory map. It's possible that you're fragmenting the C heap, by allocating variable-sized buffers. If that's the case, then you'll see an every larger virtual map, with gaps between "anon" blocks. And the solution there is to allocate constant-size blocks that are larger than you need.
这篇关于JNA / ByteBuffer的没有得到释放，造成ç堆内存耗尽的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
