Java-将哪个数据库/技术用于大量频繁更改的键值对? [英] Java - Which database/technologie to use for a huge amount of high frequently changing key-value-pairs?

问题描述

我正在开发一个Java应用程序，它正在处理大量(〜1.000.000)键-值对.键的大小是固定的，而值的大小则从20字节到最大1 KB不等.因此，在最坏的情况下，我得到了大约1 GB的数据.
键值对由多个线程(〜3-5个线程)同时读写.读写操作在大多数情况下仅影响单个键值对.
在键值对的顶部放置一个关系数据模型，该数据模型用于选择所需的键.目前，我正在使用 H2 来获取关系数据，但是我不清楚要使用哪种技术用于存储键值对.

I'm developing a Java application which is dealing with a huge amount (~ 1.000.000) of key-value-pairs. The keys have a fixed size while the size of the values vary from 20 bytes up to ~1 kbyte. So in the worst case I got something around 1 GB of data.
The key-value-pairs are read and written by multiple threads (~ 3-5 threads) concurrently. The read and write operations affect in most cases only a single key-value-pair.
On top of the key-value-pairs a relational data model is placed which is used to select the desired keys. Currently I'm using H2 for the relational data but I'm unclear on which technology to use for storing the key-value-pairs.

1. 我应使用哪个数据库存储键值对?
   • Berkeley DB 是否适合我的情况? ?

1. Which database should I use for storing the key-value-pairs?
   • Is the Berkeley DB appropriate in my case?

此外，更重要的是，我应该在哪个级别上实现同步?

Furthermore and more important, on which level should I implement synchronization?

• 数据库层对我来说似乎很明智，但是我应该如何处理应用程序内的连接管理?
• 我的第一个方法
  • 为每个读/写操作使用一个连接池和一个单独的连接.
    尽管这很容易实现，但在我看来，连接池的同步已成为瓶颈.
  • The database layer seems sensible to me but how should I deal with the connection management within the application?
  • My first appraoch
    • Using a connection pool and a separate connection for each read / write operation.
      While this is easy to implement it seems to me that the synchronization of the connection pool becomes a bottleneck.
    • 对所有读取操作使用一个连接，对所有写入操作使用另一个连接，但是我不确定该连接是否允许并发访问以及是否以可扩展的方式序列化它们.

    由于JTables定期查询键值对，因此访问时间很关键.我应该信任数据库的缓存系统还是应该在应用程序层上实现/使用另一个缓存，例如EHCache?

    Since the key-value-pairs are regularly queried by JTables the access time is critical. Should I trust on the caching system of the database or implement/use another cache on the application layer, e.g. EHCache?

    修改:
    该应用程序应在较旧的PC上运行，因此我无法将所有数据放入内存中.
    将键值对存储在现有的H2数据库中是可能的，但是将它们放入专用于键值对及其同步以提高性能的数据库中更有意义吗?
    我也不关心ACID属性.

    
    The application should run on older PCs so I can't put the whole data into memory.
    Storing the key-value-pairs in the existing H2 database would be possible but wouldn't it make more sense to put them into a database specialized for key-value-pairs and their synchronization for better performance?
    I'm also not concerned about ACID properties.

    推荐答案

    1)NoSQL数据库似乎适合您的要求:仅通过键访问值.伯克利是有序键值存储.您是否需要钥匙订购?如果没有，请检查其他解决方案:Mongo，couchbase.

    1) A NoSQL database seems suited for your requirements: accessing to values only by key. Berkeley is an ordered key-value store. Do you need order in keys? If not, check other solutions: Mongo, couchbase.

    2)在数据库级别同步将是最明智的选择.因此，我将采用您的第一种方法.您的第二种方法肯定会引起争用，并且对开发人员来说更难管理.

    2) Syncing at database level would be the most sane choice. So I would go for your first approach. Your second approach will cause contention for sure, and will be harder to manage for developers.

    3)必要时进行缓存.您是否经常访问记录? EHCache很好，您也可以使用其他系统，例如memcache.您需要根据您选择的数据库来决定缓存层.

    3) Cache if you must. Do you have frequently accessed records? EHCache is fine, you can also use other systems like memcache. You will need to decide on cache layer in line with the DB you choose.

    这篇关于Java-将哪个数据库/技术用于大量频繁更改的键值对?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！