Python:通过numpy.save保存字典 [英] Python : save dictionaries through numpy.save

问题描述

我的内存中有一个大数据集(数百万行)，以 numpy数组和 dictionaries 的形式出现.

I have a large data set (millions of rows) in memory, in the form of numpy arrays and dictionaries.

一旦构造了这些数据，我想将它们存储到文件中； 因此，以后我可以将这些文件快速加载到内存中，而无需再次从头开始重新构建这些数据.

Once this data is constructed I want to store them into files; so, later I can load these files into memory quickly, without reconstructing this data from the scratch once again.

np.save 和 np.load 函数可以使numpy数组顺利进行.
但是我面对字典对象的问题.

np.save and np.load functions does the job smoothly for numpy arrays.
But I am facing problems with dict objects.

请参阅以下示例. d2是从文件加载的字典. 请参阅#out [28]，它已作为一个numpy数组而不是作为dict被加载到d2中.因此，诸如get之类的进一步的dict操作无法正常工作.

See below sample. d2 is the dictionary which was loaded from the file. See #out[28] it has been loaded into d2 as a numpy array, not as a dict. So further dict operations such as get are not working.

是否有一种方法可以将文件中的数据作为dict(而不是numpy数组)加载?

Is there a way to load the data from the file as dict (instead of numpy array) ?

In [25]: d1={'key1':[5,10], 'key2':[50,100]}

In [26]: np.save("d1.npy", d1)

In [27]: d2=np.load("d1.npy")

In [28]: d2
Out[28]: array({'key2': [50, 100], 'key1': [5, 10]}, dtype=object)

In [30]: d1.get('key1')  #original dict before saving into file
Out[30]: [5, 10]

In [31]: d2.get('key2')  #dictionary loaded from the file
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-31-23e02e45bf22> in <module>()
----> 1 d2.get('key2')

AttributeError: 'numpy.ndarray' object has no attribute 'get'

推荐答案

这是一个结构化数组.首先使用d2.item()检索实际的dict对象:

It's a structured array. Use d2.item() to retrieve the actual dict object first:

import numpy as np

d1={'key1':[5,10], 'key2':[50,100]}
np.save("d1.npy", d1)
d2=np.load("d1.npy")
print d1.get('key1')
print d2.item().get('key2')

结果:

[5, 10]
[50, 100]

这篇关于Python:通过numpy.save保存字典的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！