无法在Python中加载先前转储的pickle文件 [英] Unable to load a previously dumped pickle file in Python

问题描述

我使用的已实现算法非常繁重，分为三个部分.因此，我使用pickle将各个阶段之间的所有内容转储，以便分别在每个阶段进行测试.

The implemented algorithm which I use is quite heavy and has three parts. Thus, I used pickle to dump everything in between various stages in order to do testing on each stage separately.

尽管第一个转储始终可以正常工作，但是第二个转储的行为似乎取决于大小.它适用于较小的数据集，但不适用于较大的数据集. (我尝试创建的热图实际上也会发生同样的情况，但这是一个不同的问题)转储的文件约为10MB，因此并没有什么大的问题.

Although the first dump always works fine, the second one behaves as if it is size dependent. It will work for a smaller dataset but not for a somewhat larger one. (The same actually also happens with a heatmap I try to create but that's a different question) The dumped file is about 10MB so it's nothing really large.

造成问题的转储包含整个类，而整个类又包含方法，字典，列表和变量.

The dump which creates the problem contains a whole class which in turn contains methods, dictionaries, lists and variables.

我实际上尝试从类的内部和外部进行转储，但是都失败了. 我正在使用的代码如下:

I actually tried dumping both from inside and outside the class but both failed. The code I'm using looks like this:

data = pickle.load(open("./data/tmp/data.pck", 'rb')) #Reads from the previous stage dump and works fine.
dataEvol = data.evol_detect(prevTimeslots, xLablNum) #Export the class to dataEvol
dataEvolPck = open("./data/tmp/dataEvol.pck", "wb") #open works fine
pickle.dump(dataEvol, dataEvolPck, protocol = 2) #dump works fine
dataEvolPck.close()

甚至尝试过:

dataPck = open("./data/tmp/dataFull.pck", "wb")
pickle.dump(self, dataPck, protocol=2) #self here is the dataEvol in the previous part of code
dataPck.close()

当我尝试使用此部分加载类时出现问题:

The problem appears when i try to load the class using this part:

dataEvol = pickle.load(open("./data/tmp/dataEvol.pck", 'rb'))

手中的错误是:

Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
dataEvol = pickle.load(open("./data/tmp/dataEvol.pck", 'rb'))
ValueError: itemsize cannot be zero

有什么想法吗? 我在64位Win-7计算机上使用Python 3.3.如果我缺少任何必要的内容，请原谅我，因为这是我的第一个问题.

Any ideas? I'm using Python 3.3 on a 64-bit Win-7 computer. Please forgive me if I'm missing anything essential as this is my first question.

答案:

推荐答案

这是一个NumPy错误，最近已在此拉动请求.要重现它，请尝试:

It is a NumPy bug that has been fixed recently in this pull request. To reproduce it, try:

import cPickle
import numpy as np
cPickle.loads(cPickle.dumps(np.string_('')))

这篇关于无法在Python中加载先前转储的pickle文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！