使用内存数据库存储数百万个临时值的效率如何？ [英] How efficient will be to use a in memory database to store millions of temporary values?

问题描述

我的应用程序目前存储数百万 Double 元素进行计算。这些值在用于计算结束时运行的特定算法之前只是临时值。完成此计算后，可以丢弃数百万个值。

My application currently stores millions of Double elements for a calculation. These values are only temporary values before they are used for a specific algorithm that is run at the end of the calculation. Once this calculation is done, the millions of values can be discarded.

完整的故事是这里。

其中一个解决方案建议的是使用内存数据库。

One of the solutions that was proposed is to use an in-memory database.

因此，如果我使用此解决方案，我将使用此数据库将我的值存储在表中以替换我的当前 Map< String，List< Double>> ，如：

So if I go with this solution, I will use this database to store my values in a table to replace my current Map<String, List<Double>>, like:

create table CALCULATION_RESULTS_XXX (
  deal_id varchar2,
  values number
);

（每个计算一个表， XXX 是计算ID）

(one table per calculation, XXX is the calculation ID)

因此在计算过程中，我将执行以下操作：

So during the calculation, I will do the following:

1. 当计算开始时，我创建了 CALCULATION_RESULTS_XXX 表。


2. 每次我需要添加一个值时，我插入一个记录在此表中。


3. 在计算结束时，我将表格内容用于我的算法。


4. 最后，我删除此表。

1. When the calculation is started, I create the CALCULATION_RESULTS_XXX table.
2. Every time I need to add a value, I insert a record in this table.
3. At the end of the calculation, I use the table content for my algorithm.
4. Finally, I drop this table.

如其他的列表Double 将需要大约240Mb。

As explained in the other subject, currently, my calculation may store several hundreds of Mb of data in the memory, as a list of 30 * 1,000,000 of Double will need about 240Mb.

现在的问题：

1. 如果我使用内存数据库，我的内存消耗是否会减少？


2. 什么关于数据库使用（或表格创建），数据插入等，我必须注意哪些具体要点？


3. 我想我会选择 H2数据库。你认为这是我需要的最佳选择吗？

1. If I go with an in-memory database, does my memory consomption will be decreased?
2. What are the specific points that I will have to take care regarding the database usage (or table creation), the data insertion, etc. ?
3. I think I will choose H2 database. Do you think it's the best choice for my needs?

推荐答案

问题很简单，你真的需要试一试，看看（性能）结果是如何运作的。

The problem is sufficiently simple that you really need to just give it a go and see how the (performance) results work out.

你已经有了一个只使用简单的内存结构的实现。就个人而言，即使戴尔最便宜的电脑配备1GB + RAM，你也可以坚持下去。除此之外，在一两个数据库中使用它应该相当简单。我会考虑Sleepycat Berkerly DB（现在由Oracle拥有...），因为你不需要使用SQL，它们应该非常高效。 （他们确实支持Java）。

You already have an implementation that just uses simple in-memory structures. Personally, given that even the cheapest computer from Dell comes with 1GB+ of RAM, you might as well stick with that. That aside, it should be fairly simple to wack in a database or two. I'd consider Sleepycat Berkerly DB (Which is now owned by Oracle...), because you don't need to use SQL and they should be quite efficient. (They do support Java).

如果结果很有希望，我会考虑进一步调查，但这真的应该只需要几天的工作，最多，包括基准测试。

If the results are promising, I'd then consider further investigation, but this really should only take a few days work, at most, including the benchmarking.

这篇关于使用内存数据库存储数百万个临时值的效率如何？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！