腌制没有__reduce__方法的字典子类不会加载成员属性 [英] Pickle a dict subclass without __reduce__ method does not load member attributes

问题描述

我需要确保dict只能接受某种类型的对象作为值.它也必须是可挑选的. 这是我的第一次尝试:

I have the need to ensure that a dict can only accept a certain type of objects as values. It also have to be pickable. Here is my first attempt:

import pickle

class TypedDict(dict):
    _dict_type = None

    def __init__(self, dict_type, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self._dict_type = dict_type

    def __setitem__(self, key, value):
        if not isinstance(value, self._dict_type):
            raise TypeError('Wrong type')
        super().__setitem__(key, value)

如果我使用以下代码(python 3.5)对其进行测试

If I test it with the following code (python 3.5)

my_dict = TypedDict(int)
my_dict['foo'] = 98

with open('out.pkl', 'wb') as fin:
    pickle.dump(my_dict, fin)

with open('out.pkl', 'rb') as fin:
    out = pickle.load(fin)

我收到错误:TypeError: isinstance() arg 2 must be a type or tuple of types.
似乎它没有为_dict_type加载正确的值，而是使用默认的None.
另外，它似乎取决于协议，好像它与protocol=0

I get the error: TypeError: isinstance() arg 2 must be a type or tuple of types.
It seems that it is not loading the correct value for _dict_type and it is instead using the default None.
Also, It seems to be dependent on the protocol as if it is working correctly with protocol=0

但是，如果我重写__reduce__方法并仅调用super，那么一切将神奇地起作用.

However, if I override the __reduce__ method and just call the super everything magically works.

def __reduce__(self):
    return super().__reduce__()

这怎么可能?这两个类(不带__reduce__)不应该等效吗?我想念什么?

How it is possible? Shouldn't be the two classes (w/o __reduce__) equivalent? What am I missing?

推荐答案

这怎么可能?这两个类(不带__reduce__)不应该等效吗?我想念什么?

How it is possible? Shouldn't be the two classes (w/o __reduce__) equivalent? What am I missing?

您错过了关键步骤:如果没有__reduce__方法(或者它失败了！)，它将使用其他方法来使您的类腌制.因此，具有__reduce__的类的行为将类似于没有__reduce__的类的行为(有几种特殊的方法具有这种行为)！

You're missing a crucial step: If there is no __reduce__ method (or if it fails!) it will use other means to pickle your class. So a class with __reduce__ won't behave like a class without __reduce__ (there are several special methods that behave like that)!

在您的第一种情况下，它将默认为基本的dict转储和加载，然后处理子类逻辑.因此它将使用几个__setitem__调用创建字典，然后设置实例属性.但是您的__setitem__需要实例属性 _dict_type.如果没有，则默认为 class属性 None，该属性会失败

In your first case it will default to basic dict dumping and loading and then handling the subclasses logic. So it will create the dictionary using several __setitem__ calls and then set the instance attributes. But your __setitem__ requires the instance attribute _dict_type. If it doesn't have one it will default to the class attribute None, which fails with the

TypeError: isinstance() arg 2 must be a type or tuple of types

这就是为什么如果您想在没有任何键值对的情况下腌制TypedDict而没有__reduce__的话，它会起作用.因为它将不会调用__setitem__并随后设置instance属性:

That's why it works if you want to pickle your TypedDict without __reduce__ if it doesn't contain any key-value pairs. Because it won't call __setitem__ and afterwards sets the instance attribute:

my_dict = TypedDict(int)

with open('out.pkl', 'wb') as fin:
    pickle.dump(my_dict, fin)

with open('out.pkl', 'rb') as fin:
    out = pickle.load(fin)

print(out._dict_type)   # int

另一方面，如果您实现__reduce__方法，则可以正常工作，因为与普通dicts会因__reduce__失败而不同-它确实适用于子类(但如果您未实现__reduce__，则不会尝试):

On the other hand it works if you implement your __reduce__ method because unlike normal dicts which fail with __reduce__ - it does work for subclasses (but it's not attempted if you don't implement __reduce__):

>>> d = {1: 1}
>>> dict.__reduce__(d)
TypeError: "can't pickle dict objects"

>>> d = TypedDict(int)
>>> dict.__reduce__(d)
(<function copyreg._reconstructor>,
 (__main__.TypedDict, dict, {}),
 {'_dict_type': int})

这篇关于腌制没有__reduce__方法的字典子类不会加载成员属性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！