使用MapMaker创建缓存 [英] Using MapMaker to create a cache
问题描述
我想使用MapMaker创建一个可缓存大对象的地图， 如果没有足够的内存，应将其从缓存中删除. 这个小演示程序似乎可以正常工作:
I want to use MapMaker to create a map that caches large objects, which should be removed from the cache if there is not enough memory. This little demo program seems to work fine:
public class TestValue {
private final int id;
private final int[] data = new int[100000];
public TestValue(int id) {
this.id = id;
}
@Override
protected void finalize() throws Throwable {
super.finalize();
System.out.println("finalized");
}
}
public class Main {
private ConcurrentMap<Integer, TestValue> cache;
MemoryMXBean memoryBean;
public Main() {
cache = new MapMaker()
.weakKeys()
.softValues()
.makeMap();
memoryBean = ManagementFactory.getMemoryMXBean();
}
public void test() {
int i = 0;
while (true) {
System.out.println("Etntries: " + cache.size() + " heap: "
+ memoryBean.getHeapMemoryUsage() + " non-heap: "
+ memoryBean.getNonHeapMemoryUsage());
for (int j = 0; j < 10; j++) {
i++;
TestValue t = new TestValue(i);
cache.put(i, t);
}
try {
Thread.sleep(100);
} catch (InterruptedException ex) {
}
}
}
/**
* @param args the command line arguments
*/
public static void main(String[] args) {
Main m = new Main();
m.test();
}
}
但是，当我在真实的应用程序中执行相同的操作时，条目是 基本上一被添加就从缓存中删除.在我的真实 应用程序中，我还使用整数作为键，并且缓存的值是 从磁盘读取的包含一些数据的存档块.据我 可以理解，弱引用一经被垃圾收集 不再使用，因此这似乎很有意义，因为按键很弱 参考.如果我这样创建地图:
However, when I do the same thing in my real application, entries are basically removed from the cache as soon as they are added. In my real application, I also use integers as keys, and the cached values are archive blocks read from disk that contains some data. As far as I understand, weak-references are garbage-collected as soon as they are no longer used, so this seems to make sense because the keys are weak references. If I create the map like this:
data = new MapMaker()
.softValues()
.makeMap();
这些条目永远不会被垃圾收集，而我的内存不足 我的测试程序中出现错误. TestValue条目上的finalize方法 从来没有被调用.如果我将测试方法更改为以下内容:
The entries are never garbage-collected and I get an out-of-memory error in my test program. The finalize method on the TestValue entries is never called. If I change the test method to the following:
public void test() {
int i = 0;
while (true) {
for (final Entry<Integer, TestValue> entry :
data.entrySet()) {
if (entry.getValue() == null) {
data.remove(entry.getKey());
}
}
System.out.println("Etntries: " + data.size() + " heap: "
+ memoryBean.getHeapMemoryUsage() + " non-heap: "
+ memoryBean.getNonHeapMemoryUsage());
for (int j = 0; j < 10; j++) {
i++;
TestValue t = new TestValue(i);
data.put(i, t);
}
try {
Thread.sleep(100);
} catch (InterruptedException ex) {
}
}
}
条目从缓存和TestValue上的终结器中删除 对象被调用，但过一会儿我也出现了内存不足的情况 错误.
entries are removed from the cache and the finalizer on the TestValue objects is called, but after a while I also get an out-of-memory error.
所以我的问题是:使用MapMaker创建地图的正确方法是什么 可以用作缓存的地图?为什么我的测试程序无法删除 如果我使用weakKeys，请尽快输入条目?是否有可能 将参考队列添加到缓存映射?
So my question is: what is the right way to use MapMaker to create a map that can be used as a cache? Why does my test program not remove the entries as soon as possible if I use weakKeys? Is it possible to add a reference queue to the cache map?
推荐答案
可能正在发生很多事情，但是对于使用软值的测试程序:即使您拥有SoftReferences，也可能会得到OutOfMemoryError尚未被垃圾收集.值得重复一遍:即使您有尚未清除的SoftReferences，您也可能会收到OutOfMemoryError.
There are a lot of things which might be going on, but with respect to your test program using soft values: you can get OutOfMemoryError even if you have SoftReferences which have not yet been garbage collected. That bears repeating: you can get an OutOfMemoryError even if you have SoftReferences which have not yet be cleared.
SoftReferences有点奇怪，请参阅 http: //jeremymanson.blogspot.com/2009/07/how-hotspot-decides-to-clear_07.html 来描述当前的机制.在您的测试用例中，GC可能没有时间做两个完整的GC.
SoftReferences are a little weird, see http://jeremymanson.blogspot.com/2009/07/how-hotspot-decides-to-clear_07.html for a description of current mechanics. Likely in your test case, the GC just didn't have time to do two full GCs.
当您使用weakKeys时，CG会立即清除它们，而不必等待整个GC暂停. (b/c WeakReferences是积极收集的.)
When you were using weakKeys, the CG cleared them right away, and didn't have to wait for a full GC pause. (b/c WeakReferences are collected aggressively.)
我认为，如果您希望使用具有Integer键的内存敏感型缓存，我认为以下情况是合适的:
In my opinion, if you want a memory-sensitive cache with Integer keys, I'd think the following is appropriate:
data = new MapMaker().softValues().makeMap();
您可以轻松地创建一个抛出OutOfMemoryError的测试程序，但是如果您的实际应用程序表现良好，并且不会承受太大的压力，则可能会好的. SoftReferences很难正确设置.
You can easily make a test program which throws OutOfMemoryError, but if your real application is somewhat well behaved, and doesn't get under too much pressure, you might be OK. SoftReferences are pretty hard to get right.
如果需要使用System.gc()避免内存不足，我建议您切换到具有固定最大大小的LRU映射(有关示例，请参见java.util.LinkedHashMap的javadoc.)它不是并发的，但是我希望它最终可以为您提供比要求系统额外花费大量时间进行全暂停垃圾收集的吞吐量.
If you need to use System.gc() avoid out-of-memory, I would instead recommend you switch to an LRU map with a fixed max size (See the javadoc of java.util.LinkedHashMap for an example.) It's not concurrent, but I expect it's going to give you better throughput in the end than asking the system to do a full-pause garbage collection a bunch of extra times.
哦，还有关于整数键和weakKeys()的最后一点:MapMaker在使用弱键或软键时对键使用身份比较，这很难正确完成.见证以下内容:
Oh, and a final note about integer keys and weakKeys(): MapMaker uses identity comparison for keys when using weak or soft keys, and that's pretty hard to do correctly. Witness the following:
Map<Integer,String> map = new MapMaker().weakKeys().makeMap();
Integer a = new Integer(1);
Integer b = new Integer(1);
Integer c = 1; //auto box
Integer d = 1; //auto box
map.put(a, "A");
map.put(b, "B");
map.put(c,"C");
map.put(d,"D");
map.size() // size is 3;
祝你好运.
这篇关于使用MapMaker创建缓存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
