NoSQL 和气象数据 [英] NoSQL and meteorological data

问题描述

所以有一个很酷的新东西，这些 NoSQL 数据库.这就是我的数据: 气象数据行的行:值，代表某个站点(由 WMO 编号，而不是坐标标识)在某个时间的某些测量值.

So there's this new cool thing, these NoSQL-databases. And so there's my data: Rows of rows of rows of meteorological data: Values, representing certain measurements at a certain station (Identified by a WMO number, not coordinates), at a certain time.

不是每个站点都测量每个参数，也不是所有参数都一直被测量.

Not every station measures every parameter, not every parameter is measured all the time.

我目前在 MySQL 中存储这些数据(每小时价值 30 年，产生约 10 亿个价值).持续的增长和可预见的更多数据的增加让我有点头疼.

I store this data (30 years worth of hourly values, resulting in ~1 billion values) currently in MySQL. The continous growth and the forseeable addition of even more data give me a little headache.

阅读基于文档的 NoSQL 系统似乎很容易扩展，我想知道 NoSQL 是否也是气象数据的可行数据存储概念.你有这方面的经验吗?

Reading about the document based NoSQL systems which seem to scale rather easily, I was wondering if NoSQL is a viable data storage concept for meteorological data too. Do you have any experience with this?

更新:忘记了典型查询:大多数查询需要时间轴中的数据:即给我 066310 站从 01.01.2010 00:00 到 01.03.2010 00:00 的温度.

Update: Forgot about typical queries: Most of the queries need data in the temporal axis: I.e. give me the temperatures of station 066310 from 01.01.2010 00:00 to 01.03.2010 00:00.

或者:给我一个特定站的所有参数的最新值.

Or: give me the most recent values of all parameters of a particular station.

推荐答案

当您的数据结构非常简单(例如一个简单的键值存储)/可预测并且您不需要关系完整性或需要临时和/或高级查询.

NoSQL could be a fit when your data structure is quite simple (for example a simple key-value store) / predictable and you have no need for relational integrity or a need for ad-hoc and/or advanced querying.

您在轻松可扩展性方面的优势可能会在灵活性和一致性方面有所下降.

What you win in easy scalability you might lose in flexibility and consistency though.

最大的问题是有一种简单的方法来对您的数据进行复杂的查询.我会说计量数据不是 NoSQL 的最佳候选.

The biggest problem would be to have an easy means for composing complex queries over your data. I would say meterological data is not the best candidate for NoSQL.

我个人更喜欢 PostgreSQL 而不是 MySQL，并且发现它在正确设置时非常具有可扩展性(即使有数百万甚至数十亿行).

I personally prefer PostgreSQL over MySQL and find it very scalable (even with millions or even billions of rows) when setup correctly.

这篇关于NoSQL 和气象数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！