键值存储建议 [英] key-value store suggestion

问题描述

我需要一个非常基本的 Java 键值对存储.我从 HashMap 开始，但似乎 HashMap 有点空间效率低下(我存储了大约 2000 万条记录，并且似乎需要大约 6GB RAM).

I need a very basic key-value store for java. I started with a HashMap but it seems that HashMap is somewhat space inefficient (I'm storing ~20 million records, and seems to require ~6GB RAM).

地图是 Map，所以我正在考虑使用 GNU Trove TIntObjectHashMap，并将地图值存储为一个 ascii 字节数组而不是字符串.

The map is Map<Integer,String>, and so I'm considering using GNU Trove TIntObjectHashMap<byte[]>, and storing the map value as an ascii byte array rather than String.

作为替代方案，是否有一个键值存储只需要添加 jar 文件，不会一次将整个映射保存在 RAM 中，并且仍然相当快?

As an alternative to that, is there a key-value store that only requires adding jar files, does not hold the entire map in RAM at once, and is still reasonably fast?

推荐答案

使用伯克利数据库.

Berkeley DB 将对象图、集合中的对象或简单的二进制键/值数据直接存储在磁盘上的 btree 中.这种简单、高效的方法消除了 ORM 解决方案中所有不必要的开销.使用直接持久层 (DPL) Java 开发人员使用存储信息对类进行注释，这与 JPA 非常相似.这种方法熟悉、高效且快速.DPL 在不牺牲速度的情况下降低了数据存储的复杂性.

Berkeley DB stores object graphs, objects in collections, or simple binary key/value data directly in an a btree on disk. This simple, highly efficient approach removes all the unnecessary overhead in ORM solutions. Using the Direct Persistence Layer (DPL) Java developers annotate classes with storage information, much like JPA. This approach is familiar, efficient, and fast. The DPL reduces the complexity of data storage while not sacrificing speed.

这肯定会给您带来内存和速度方面的巨大收益，同时不会增加应用程序的复杂性.享受吧！

This should definitely give you huge gains in memory and speed, while not increasing the complexity of your application. Enjoy!

这篇关于键值存储建议的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！