无法使用递归引用腌制对象 [英] Can't pickle objects with recursive references

问题描述

我想创建一组这些对象并腌制/取消腌制它们.但是，如果存在递归引用，例如:

I want to create a set of these objects and pickle/unpickle them. However, if there is a recursive reference such as:

mdp1 = MDP(0, (0,1))
mdp2 = MDP(0, (0,2))
mdp1.adj = frozenset({mdp2})
mdp2.adj = frozenset({mdp1})
mdps = {mdp1, mdp2}
p_on = open("test.pickle", "wb")
pickle.dump(mdps, p_on)
p_on.close()

p_off = open("test.pickle", "rb")
emp = pickle.load(p_off)
print(emp)

在这种情况下，mdp1.adj 包含 mdp2，mdp2 包含 mdp1.

In this case mdp1.adj contains mdp2 and mdp2 contains mdp1.

作为最小的可重复示例:

As a minimal reproducible example:

import pickle


    class MDP(object):
        def __init__(self, level, state_var):
            self.state_var = state_var
            self.level = level
            self.adj = frozenset()
            self.mer = frozenset()
            ...

        def __repr__(self):
           return "level {} var {} mer {}".format(self.level, self.state_var, self.mer)
    
        def __eq__(self, other):
            if isinstance(other, MDP):
                return (self.level == other.level and self.state_var == other.state_var and self.mer==other.mer)
            else:
                return False
    
        def __hash__(self):
            return hash(self.__repr__())
    
        def __lt__(self, other):
            return self.state_var < other.state_var

    def main():
        mdp1 = MDP(0, (0,1))
        mdp2 = MDP(0, (0,2))
        mdp1.adj = frozenset({mdp2})
        mdp2.adj = frozenset({mdp1})
        mdps = {mdp1, mdp2}
        p_on = open("test.pickle", "wb")
        pickle.dump(mdps, p_on)
        p_on.close()

        p_off = open("test.pickle", "rb")
        emp = pickle.load(p_off)
        print(emp)

    main()

我得到以下堆栈跟踪:

Traceback (most recent call last):
  File "../test.py", line 43, in <module>
    main()
  File "../test.py", line 39, in main
    emp = pickle.load(p_off)
  File "../test.py", line 22, in __hash__
    return hash(self.__repr__())
  File "../test.py", line 13, in __repr__
    return "level {} var {} mer {}".format(self.level, self.state_var, self.mer)
AttributeError: 'MDP' object has no attribute 'level'

推荐答案

对于任何有同样问题的人 user2357112 帮助解释了在 unpickling 过程中:由于循环引用，一些对象将不得不在 unpickling 过程中看到其他处于未初始化或部分初始化状态的对象，而 pickle 不知道是什么情况或不是什么情况t 安全.它最终会尝试在设置级别之前将 MDP 实例添加到 adj 集.

For anyone with the same question user2357112 helped explain that during the unpickling process: due to the circular references, some objects are going to have to see other objects in uninitialized or partially-initialized states during the unpickling process, and pickle has no idea what cases that is or isn't safe for. It ends up trying to add an MDP instance to an adj set before setting level.

发生这种情况是因为我在类中定义散列函数和等式的方式.删除那些自定义定义并让 python 处理相等和散列解决了我的问题.Pickle 能够使用循环引用序列化对象:https://pymotw.com/2/pickle/#circular-references

This happens because of the way I've defined hash functions and equality in my class. Deleting those custom definitions and letting python handle equality and hashing fixed my problem. Pickle is capable of serializing objects with circular references: https://pymotw.com/2/pickle/#circular-references

这篇关于无法使用递归引用腌制对象的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！