cPickle:SystemError:错误返回而未设置异常 [英] cPickle:SystemError: error return without exception set

问题描述

在按如下方式转储大型矩阵(170000 * 20000)时

While dumping large matrix (170000*20000) as follows

 cPickle.dump(train_set,gzip.open('train.pickle.gz','wb'), cPickle.HIGHEST_PROTOCOL)

我收到以下错误:

SystemError: error return without exception set

在这种情况下我该如何处理?

How would I deal in this case?

推荐答案

cPickle不能用于存储非常大的对象(请参见 http://bugs.python.org/issue11564 ).

cPickle can't be used for storing very large objects ( see http://bugs.python.org/issue11564 ).

您有几种选择:

• 将数据拆分为多个块并将其存储在多个文件中
• numpy.save
• h5py <-我的最爱，因为它具有方便的类似numpy的界面，而且数据存储在hdf5可以与其他软件/语言进行互操作.
• pytables

• split the data into chunks and store it in multiple files
• numpy.save
• h5py <- my favorite because of its convenient numpy-like interface, plus the data are stored in hdf5 which enables interopability with other software/languages.
• pytables

这篇关于cPickle:SystemError:错误返回而未设置异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！