python multiprocessing pickle协议 [英] python multiprocessing pickle protocol

问题描述

我正在使用Python多处理模块将对象放入队列中，并由多个工作人员对其进行处理.我的第一个问题是将绑定实例方法绑定到我一直在工作的pickle，但是现在由于对象正在使用__slots__的事实，我遇到了一个单独的问题.

I am using the Python multiprocessing module to place objects onto a queue and have them processed by several workers. My first issue was getting bound instance methods to pickle, which I have working, but now I'm running into a separate issue caused by the fact that the objects are using __slots__.

当mp模块去腌制对象时，似乎正在使用较旧的ascii腌制协议，该协议无法处理__slots__.较新的协议可以解决此问题，但是我不确定如何使mp模块使用此协议.

When the mp module goes to pickle the objects, it seems to be using the older ascii pickle protocol, that can't handle __slots__. The newer protocol does handle this, but I'm not sure how to make the mp module use this protocol.

任何人都对此有经验吗?

Anyone have any experience with this?

推荐答案

如果无法更改多处理程序包使用的pickle协议，则为对象定义__getstate__和__setstate__:

If it's not possible to change the pickle protocol the multiprocessing package uses, then define __getstate__ and __setstate__ for your objects:

import pickle

class Foo(object):
    __slots__ = ['this', 'that', 'other']

    def __init__(self):
        self.this = 1
        self.that = 2
        self.other = 3

    def __getstate__(self):
        return dict((name, getattr(self, name))
                    for name in self.__slots__)

    def __setstate__(self, state):
        for name, value in state.items():
            setattr(self, name, value)

pickle.dumps(Foo(), protocol=0)

这篇关于python multiprocessing pickle协议的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！