如何在大型对象的Python Pickle中查找错误源 [英] How to find source of error in Python Pickle on massive object

问题描述

我已经为一个相当大的项目接管了某人的代码.我正在尝试保存程序状态，并且有一个庞大的对象可以存储几乎所有其他对象.我正在尝试腌制该对象，但出现此错误:

I've taken over somebody's code for a fairly large project. I'm trying to save program state, and there's one massive object which stores pretty much all the other objects. I'm trying to pickle this object, but I get this error:

pickle.PicklingError:无法腌制:找不到为内置 .module

pickle.PicklingError: Can't pickle : it's not found as builtin.module

从我在Google上可以找到的信息，这是因为我正在某处导入python init之外的内容，或者是class属性引用了一个模块.所以，我有两个问题:

From what I can find on google, this is because somewhere I'm importing something outside of python init, or that a class attribute is referencing a module. So, I've got a two questions:

1. 任何人都可以确认这就是为什么出现此错误的原因吗?我在代码中寻找正确的东西吗?

1. Can anybody confirm that that's why this error is being given? Am I looking for the right things in my code?

是否可以找到导致腌制困难的代码/对象成员的行?追溯仅提供出现错误的pickle中的行，而不给出被腌制对象的行.

Is there a way to find what line of code/object member is causing the difficulties in pickle? The traceback only gives the line in pickle where the error occurs, not the line of the object being pickled.

推荐答案

2)您可以对pickle.Pickler和Monkey-patch进行子类化，以显示其腌制的日志.这样应该可以更容易地找到问题所在.

2) You can subclass pickle.Pickler and monkey-patch it to show a log of what it's pickling. This should make it easier to trace where the problem is.

import pickle
class MyPickler (pickle.Pickler):
    def save(self, obj):
        print 'pickling object', obj, 'of type', type(obj)
        pickle.Pickler.save(self, obj)

这仅适用于pickle.Pickler的Python实现.在Python 3.x中，pickle模块默认使用C实现，Pickler的纯Python版本称为_Pickler.

This will only work with the Python implementation of pickle.Pickler. In Python 3.x, the pickle module uses the C implementation by default, the pure-Python version of Pickler is called _Pickler.

# Python 3.x
import pickle
class MyPickler (pickle._Pickler):
    def save(self, obj):
        print ('pickling object  {0} of type {1}'.format(obj, type(obj))
        pickle._Pickler.save(self, obj)

这篇关于如何在大型对象的Python Pickle中查找错误源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！