寻找一个java.util.Map的替代品 [英] Looking for a drop-in replacement for a java.util.Map

问题描述

问题

继续这个问题，似乎基于文件或磁盘的 Map 实现可能是正确的解决我在那里提到的问题。短版：

• 现在，我有一个 Map 实现为 ConcurrentHashMap 。


• 以不变的速度持续添加条目。最后，无论如何，这意味着JVM将耗尽堆空间。

在工作中，（强烈地）建议我使用SQLite解决这个问题，但在提出上一个问题之后，我认为数据库不是这个工作的正确工具。所以 - 让我知道这是否听起来很疯狂 - 我认为更好的解决方案是存储在磁盘上的 Map 。

糟糕的想法：自己实现这一点。更好的主意：使用别人的图书馆！ 哪一个？

要求

必备品：

• 免费


• 持续。坚持在JVM重新启动之间。


• 某种可搜索性。是的，我需要能够检索此darn数据以及将其删除。基本结果集过滤是一个优势。


• 独立于平台。
需要在Windows或Linux计算机上进行生产部署。

• 的可吹扫即可。磁盘空间是有限的，就像堆空间一样。我需要摆脱 n 天之前的条目。这不是什么大问题，如果我必须手动执行此操作。

不错：

• 易于使用如果我能在本周末之前完成这项工作，那将非常棒。
  
  Better仍然：一天结束。如果我可以将一个JAR添加到我的类路径中，将新的ConcurrentHashMap< Foo，Bar>（）; 更改为 code> new SomeDiskStoredMap< Foo，Bar>（）;
  
  并完成。


• 体面可伸缩性性能。最糟糕的情况：每天，每秒，每秒，每天都会添加新条目（平均）。但是，插入并不总是顺利发生。它可能是（没有插入一小时）然后（一次插入10,000个对象）
$




可能的解决方案






• Ehcache ？我从来没有用过它。这是一个建议解决方案到我以前的问题。


• Berkeley DB ？再一次，我从来没有使用它，我真的不知道任何关于它。
Hadoop （以及哪个子项目）？没有用过它。根据这些文档，其跨平台准备就不明确我。在可预见的将来，我不需要分布式操作。 SQLite JDBC驱动程序毕竟是什么？

• ???




Ehcache和Berkeley DB现在看起来都很合理。任何方向的任何特定建议？

解决方案

更新（在第一篇文章后的大约4年...）：请注意，的ehcache，高速缓存项目的持久性仅在付费产品中可用。感谢@boday指出了这一点。





ehcache非常棒。它将为您提供实现地图内存，磁盘或内存溢出到磁盘所需的灵活性。如果你为java.util.Map使用这个非常简单的包装，那么使用它就非常简单：



  import java.util.Collection ; 
 import java.util.List; 
 import java.util.Map; 
 import java.util.Set; 
 
导入net.sf.ehcache.Cache; 
导入net.sf.ehcache.Element; 
 
导入org.apache.log4j.Logger; 
 
导入com.google.common.collect.Sets; 
 
公共类EhCacheMapAdapter< K，V>实现Map< K，V> {
 @SuppressWarnings（unused）
私有最终静态记录器记录器=记录器
 .getLogger（EhCacheMapAdapter.class）; 
 
 public Cache ehCache; 
 
 public EhCacheMapAdapter（Cache ehCache）{
 super（）; 
 this.ehCache = ehCache; 
} //结束构造函数
 
 @Override 
 public void clear（）{
 ehCache.removeAll（）; 
} //结束方法
 
 @Override 
 public boolean containsKey（Object key）{
 return ehCache.isKeyInCache（key）; 
} //结束方法
 
 @Override 
 public boolean containsValue（Object value）{
 return ehCache.isValueInCache（value）; 
} //结束方法
 
 @Override 
 public Set< Entry< K，V>> entrySet（）{
 throw new UnsupportedOperationException（）; 
} //结束方法
 
 @SuppressWarnings（unchecked）
 @Override 
 public V get（Object key）{
 if（key = = null）返回null; 
 Element element = ehCache.get（key）; 
 if（element == null）return null; 
 return（V）element.getObjectValue（）; 
} //结束方法
 
 @Override 
 public boolean isEmpty（）{
 return ehCache.getSize（）== 0; 
} //结束方法
 
 @SuppressWarnings（unchecked）
 @Override 
 public Set< K> keySet（）{
列表< K> l = ehCache.getKeys（）; 
返回Sets.newHashSet（l）; 
} //结束方法
 
 @SuppressWarnings（unchecked）
 @Override 
 public V put（K key，V value）{
 Object o = this.get（key）; 
 if（o！= null）return（V）o; 
 Element e = new Element（key，value）; 
 ehCache.put（e）; 
返回null; 
} //结束方法
 
 
 @Override 
 public V remove（Object key）{
 V retObj = null; 
 if（this.containsKey（key））{
 retObj = this.get（key）; 
} // end if 
 ehCache.remove（key）; 
返回retObj; 
} //结束方法
 
 @Override 
 public int size（）{
 return ehCache.getSize（）; 
} //结束方法
 
 @覆盖
 public Collection< V> values（）{
 throw new UnsupportedOperationException（）; 
} //结束方法
 $ b $ @覆盖
 public void putAll（Map <？extends K，？extends V> m）{
 for（K key：m .keySet（））{
 this.put（key，m.get（key））; 
} // end for 
} // end method 
} // end class 
  




Problem

Following up on this question, it seems that a file- or disk-based Map implementation may be the right solution to the problems I mentioned there. Short version:

• Right now, I have a Map implemented as a ConcurrentHashMap.
• Entries are added to it continually, at a fairly fixed rate. Details on this later.
• Eventually, no matter what, this means the JVM runs out of heap space.

At work, it was (strongly) suggested that I solve this problem using SQLite, but after asking that previous question, I don't think that a database is the right tool for this job. So - let me know if this sounds crazy - I think a better solution would be a Map stored on disk.

Bad idea: implement this myself. Better idea: use someone else's library! Which one?

Requirements

Must-haves:

• Free.
• Persistent. The data needs to stick around between JVM restarts.
• Some sort of searchability. Yes, I need the ability to retrieve this darn data as well as put it away. Basic result set filtering is a plus.
• Platform-independent. Needs to be production-deployable on Windows or Linux machines.
• Purgeable. Disk space is finite, just like heap space. I need to get rid of entries that are n days old. It's not a big deal if I have to do this manually.

Nice-to-haves:

• Easy to use. It would be great if I could get this working by the end of the week.
  Better still: the end of the day. It would be really, really great if I could add one JAR to my classpath, change new ConcurrentHashMap<Foo, Bar>(); to new SomeDiskStoredMap<Foo, Bar>();
  and be done.
• Decent scalability and performance. Worst case: new entries are added (on average) 3 times per second, every second, all day long, every day. However, inserts won't always happen that smoothly. It might be (no inserts for an hour) then (insert 10,000 objects at once).

Possible Solutions

• Ehcache? I've never used it before. It was a suggested solution to my previous question.
• Berkeley DB? Again, I've never used it, and I really don't know anything about it.
• Hadoop (and which subproject)? Haven't used it. Based on these docs, its cross-platform-readiness is ambiguous to me. I don't need distributed operation in the foreseeable future.
• A SQLite JDBC driver after all?
• ???

Ehcache and Berkeley DB both look reasonable right now. Any particular recommendations in either direction?

解决方案

UPDATE (some 4 years after first post...): beware that in newer versions of ehcache, persistence of cache items is available only in the pay product. Thanks @boday for pointing this out.

ehcache is great. It will give you the flexibility you need to implement the map in memory, disk or memory with spillover to disk. If you use this very simple wrapper for java.util.Map then using it is blindingly simple:

import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Set;

import net.sf.ehcache.Cache;
import net.sf.ehcache.Element;

import org.apache.log4j.Logger;

import com.google.common.collect.Sets;

public class EhCacheMapAdapter<K,V> implements Map<K,V> {
    @SuppressWarnings("unused")
    private final static Logger logger = Logger
            .getLogger(EhCacheMapAdapter.class);

    public Cache ehCache;

    public EhCacheMapAdapter(Cache ehCache) {
        super();
        this.ehCache = ehCache;
    } // end constructor

    @Override
    public void clear() {
        ehCache.removeAll();
    } // end method

    @Override
    public boolean containsKey(Object key) {
        return ehCache.isKeyInCache(key);
    } // end method

    @Override
    public boolean containsValue(Object value) {
        return ehCache.isValueInCache(value);
    } // end method

    @Override
    public Set<Entry<K, V>> entrySet() {
        throw new UnsupportedOperationException();
    } // end method

    @SuppressWarnings("unchecked")
    @Override
    public V get(Object key) {
        if( key == null ) return null;
        Element element = ehCache.get(key);
        if( element == null ) return null;
        return (V)element.getObjectValue();
    } // end method

    @Override
    public boolean isEmpty() {
        return ehCache.getSize() == 0;
    } // end method

    @SuppressWarnings("unchecked")
    @Override
    public Set<K> keySet() {
        List<K> l = ehCache.getKeys();
        return Sets.newHashSet(l);
    } // end method

    @SuppressWarnings("unchecked")
    @Override
    public V put(K key, V value) {
        Object o = this.get(key);
        if( o != null ) return (V)o;
        Element e = new Element(key,value);
        ehCache.put(e);
        return null;
    } // end method


    @Override
    public V remove(Object key) {
        V retObj = null;
        if( this.containsKey(key) ) {
            retObj = this.get(key);
        } // end if
        ehCache.remove(key);
        return retObj;
    } // end method

    @Override
    public int size() {
        return ehCache.getSize();
    } // end method

    @Override
    public Collection<V> values() {
        throw new UnsupportedOperationException();
    } // end method

    @Override
    public void putAll(Map<? extends K, ? extends V> m) {
        for( K key : m.keySet() ) {
            this.put(key, m.get(key));
        } // end for
    } // end method
} // end class

这篇关于寻找一个java.util.Map的替代品的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！