高效的数据结构，用于存储N个很大的N个列表 [英] Efficient data structure for storing N lists where N is very large

问题描述

我将需要存储N个列表，其中N大(一百万).例如，

I will need to store N lists, where N is large (1 million). For example,

[2,3]
[4,5,6]
...
[4,5,6,7]

每个项目都是大约0-10000个元素的列表.我想使用一个列表的numpy数组，例如

Each item is a list of about 0-10000 elements. I wanted to use a numpy array of lists, like

np.array([[2,3],[4,5,6])

然后，尝试附加到numpy数组中的列表时遇到效率问题.在这里我也被告知:

Then I got efficiency issues when trying to append to the lists in the numpy array. Also I was told here: Efficiently append an element to each of the lists in a large numpy array, to not use numpy array of lists.

就内存和时间效率而言，用于存储此类数据的良好数据结构是什么?

What would be a good data structure for storing such data, in terms of memory and time efficiency?

推荐答案

除非您存储的元素遵循某种模式，否则您必须使用嵌套列表，因为没有其他方法可以使这些元素脱颖而出.

Unless the elements youre storing follow some pattern you must use nested list since there is no other way to get those elements out of the others.

在Python中:

listOfLists = [[1,2,3],
               [4,5,6],
               [7,8,9]]

因此，每当您要使用此列表进行操作时，都可以使用numpy函数

So whenever you want to operate with this list you can use numpy functions

>>> np.mean(listOfLists)
5.0
>>> np.max(listOfLists)
9

这篇关于高效的数据结构，用于存储N个很大的N个列表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！