推荐快速可扩展持久化Map - Java [英] Recommend a fast & scalable persistent Map - Java

问题描述

我需要一个磁盘支持的Map结构来在Java应用程序中使用。它必须具有以下标准：

I need a disk backed Map structure to use in a Java app. It must have the following criteria:

1. 能够存储数百万条记录（甚至数十亿）


2. 快速查找 - 地图上的大部分操作将简单地查看密钥是否已经存在。这和上面的1是最重要的标准。应该有一个有效的内存缓存机制，用于经常使用的密钥。


3. 持久但不需要事务性，可能会遇到一些失败。即可快速与磁盘同步，不需要进行事务处理。


4. 能够存储简单的基本类型，但不需要存储序列化对象。


5. 它不需要分发，即将在一台机器上运行。


6. 简单的设置&


7. 不需要关系查询

1. Capable of storing millions of records (even billions)
2. Fast lookup - the majority of operations on the Map will simply to see if a key already exists. This, and 1 above are the most important criteria. There should be an effective in memory caching mechanism for frequently used keys.
3. Persistent, but does not need to be transactional, can live with some failure. i.e. happy to synch with disk periodically, and does not need to be transactional.
4. Capable of storing simple primitive types - but I don't need to store serialised objects.
5. It does not need to be distributed, i.e. will run all on one machine.
6. Simple to set up & free to use.
7. No relational queries required

记录键将是字符串或长整型。如上所述，读取将比写入更频繁，并且大多数读取将仅仅是检查密钥是否存在（即不需要读取密钥关联数据）。每个记录只会更新一次，记录不会被删除。

Records keys will be strings or longs. As described above reads will be much more frequent than writes, and the majority of reads will simply be to check if a key exists (i.e. will not need to read the keys associated data). Each record will be updated once only and records are not deleted.

我目前使用Bdb JE，但正在寻找其他选项。

I currently use Bdb JE but am seeking other options.

更新

自从我改进了现有BDB设置的查询性能后，对次级键的依赖。一些查询需要两个辅助键上的连接，并将它们组合成一个复合键，我删除了查找中的间接级别，可以很好地加快速度。

Have since improved query performance on my existing BDB setup by reducing the dependency on secondary keys. Some queries required a join on two secondary keys and by combining them into a composite key I removed a level of indirection in the lookup which speeds things up nicely.

推荐答案

我可能会使用本地数据库。就像 Bdb JE 或 HSQLDB 。我可以问这种方法有什么问题吗？你必须有一些理由来寻找替代方案。

I'd likely use a local database. Like say Bdb JE or HSQLDB. May I ask what is wrong with this approach? You must have some reason to be looking for alternatives.

回应评论：
作为问题的性能，我猜你已经在使用JDBC来处理这个可能值得尝试HSQLB并阅读有关内存和磁盘的章节使用。

这篇关于推荐快速可扩展持久化Map - Java的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！