Java处理大量数据 [英] Java handling large amounts of data

问题描述

我有一个需要显示大量数据（大约100万个数据点）的Java应用程序。数据并非都需要同时显示，而是仅当用户请求时才显示。该应用程序是一个不与应用程序服务器一起运行或打开任何集中式数据库的桌面应用程序。

I have a Java application that needs to display large amounts of data (on the order of 1 million data points). The data doesn't all need to be displayed at the same time but rather only when requested by a user. The app is a desktop app that is not running with an app server or hitting any centralized database.

我的想法是在机器上运行数据库并加载数据那里。数据库将只在大多数时间读取，所以我应该能够索引帮助优化查询。如果我在本地系统上运行，我不确定是否应该尝试并实现一些缓存（我不知道查询将运行多快，我目前正在处理它们）。

My thought was to run a database on the machine and load the data in there. The DB will be read only most of the time, so I should be able to index to help optimize queries. If I'm running on a local system, I'm not sure if I should try and implement some caching (I'm not sure how fast the queries will run, I'm currently working on them).

这是一个合理的方法来处理问题，还是会有更好的方法？

Is this is a logical way to approach the problem or would there be better approaches?

/ p>

Thanks, Jeff

推荐答案

显示和数据是两个不同的东西。

Display and data are two different things.

您不会提供任何详细信息，但可以在后台生成显示，一次导入一个数据片，然后显示时，它准备好了。许多任何东西都可能会导致内存问题，所以你需要小心。数据库将帮助持久的东西，但它不会帮助你得到十磅的数据到你的五磅记忆袋。

You don't give any details about either, but it could be possible to generate the display in the background, bringing in the data one slice at a time, and then displaying when it's ready. Lots of anything could cause memory issues, so you'll need to be careful. The database will help persist things, but it won't help you get ten pounds of data into your five pound memory bag.

更新：如果个人只读几个点，一个时间，显示不是一个问题，那么我会说，任何数据库将能够处理它，如果你索引表适当。 100万行对于有能力的数据库来说不是很多。

UPDATE: If individuals are only reading a few points at a time, and display isn't an issue, then I'd say that any database will be able to handle it if you index the table appropriately. One million rows isn't a lot for a capable database.

这篇关于Java处理大量数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！