用于大数据的数据结构 [英] What data structure to use for big data

问题描述

我有一张包含一百万行的Excel表格。每行有100列。
每行代表一个具有100个属性的类的实例，列值是这些属性的值。

I have an excel sheet with a million rows. Each row has 100 columns. Each row represents an instance of a class with 100 attributes, and the columns values are the values for these attributes.

哪种数据结构最适合在这里用来存储百万个数据实例？

What data structure is the most optimal for use here, to store the million instance of data?

谢谢

推荐答案

这实际上取决于您需要如何访问这些数据以及您想要优化的内容 - 例如，空间与速度。

It really depends on how you need to access this data and what you want to optimize for – like, space vs. speed.

• 如果你想优化空间，你可以只序列化和压缩数据，但如果你需要读取/操作数据，这可能是没用的。


• 如果你访问的话索引，最简单的是一个数组数组。


• 如果您改为使用一个对象数组，其中每个对象包含您的100个属性，您可以更好地构造代码（封装！）


• 如果您需要查询/搜索数据，它实际上取决于查询的类型。您可能想看看BST数据结构......

• If you want to optimize for space, well, you could just serialize and compress the data, but that would likely be useless if you need to read/manipulate the data.
• If you access by index, the simplest thing is an array of arrays.
• If you instead use an array of objects, where each object holds your 100 attributes, you have a better way to structure your code (encapsulation!)
• If you need to query/search the data, it really depends on the kind of queries. You may want to have a look at BST data structures...

这篇关于用于大数据的数据结构的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！