Java大数据结构用于存储矩阵 [英] Java large datastructure for storing a matrix

问题描述

我需要存储一个2d矩阵，其中包含邮政编码以及每个邮政编码之间的距离(以km为单位).我的客户有一个计算距离的应用程序，然后将其存储在Excel文件中.目前，有952个地方.因此矩阵将具有952x952 = 906304个条目.

I need to store a 2d matrix containing zip codes and the distance in km between each one of them. My client has an application that calculates the distances which are then stored in an Excel file. Currently, there are 952 places. So the matrix would have 952x952 = 906304 entries.

我试图将其映射到HashMap [Integer，Float].整数是两个地方的两个字符串的哈希码，例如"A"和"B".浮点值是它们之间的距离，以公里为单位.

I tried to map this into a HashMap[Integer, Float]. The Integer is the hash code of the two Strings for two places, e.g. "A" and "B". The float value is the distance in km between them.

在填充数据时，我在205k个条目之后遇到了OutOfMemoryExceptions.您是否有技巧提示我如何巧妙地存储它?我什至不知道将全部内存存储在内存中是否很聪明.我的选择是SQL和MS Access ...

While filling in the data I run into OutOfMemoryExceptions after 205k entries. Do you have a tip how I can store this in a clever way? I even don't know if it's clever to have the whole bunch in memory. My options are SQL and MS Access...

问题在于我需要非常快速且可能非常频繁地访问数据，这就是为什么我选择HashMap的原因，因为它在O(1)中运行以进行查找.

The problem is that I need to access the data very quickly and possibly very often which is why I chose the HashMap because it runs in O(1) for the look up.

感谢您的答复和建议！

马可

推荐答案

您能简单地增加JVM可用的内存吗?

Can you simply boost the memory available to the JVM ?

java -Xmx512m ...

通过默认，最大内存配置为64Mb. 此处.如果可以执行此操作，则可以保持数据处于处理中，并最大化性能(即，您无需即时计算).

By default the maximum memory configuration is 64Mb. Some more tuning tips here. If you can do this then you can keep the data in-process and maximise the performance (i.e. you don't need to calculate on the fly).

这篇关于Java大数据结构用于存储矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！