多处理忽略“__setstate__"； [英] multiprocessing ignores "__setstate__"

问题描述

我假设多处理包使用 pickle 在进程之间发送东西.但是，pickle 关注对象的 __getstate__ 和 __setstate__ 方法.多处理似乎忽略了它们.这样对吗?我糊涂了吗?

I assumed that the multiprocessing package used pickle to send things between processes. However, pickle pays attention to the __getstate__ and __setstate__ methods of an object. Multiprocessing seems to ignore them. Is this correct? Am I confused?

要复制，安装 docker，然后在命令行中输入

To replicate, install docker, and type into command line

$ docker run python:3.4 python -c "import pickle
import multiprocessing
import os

class Tricky:
    def __init__(self,x):
        self.data=x

    def __setstate__(self,d):
        self.data=10

    def __getstate__(self):
        return {}

def report(ar,q):
    print('running report in pid %d, hailing from %d'%(os.getpid(),os.getppid()))
    q.put(ar.data)

print('module loaded in pid %d, hailing from pid %d'%(os.getpid(),os.getppid()))
if __name__ == '__main__':
    print('hello from pid %d'%os.getpid())
    ar = Tricky(5)
    q = multiprocessing.Queue()
    p = multiprocessing.Process(target=report, args=(ar, q))
    p.start()
    p.join()
    print(q.get())
    print(pickle.loads(pickle.dumps(ar)).data)"

你应该得到类似的东西

module loaded in pid 1, hailing from pid 0
hello from pid 1
running report in pid 5, hailing from 1
5
10

我原以为是10"10"，但实际上是5"10".这意味着什么?

I would have thought it would have been "10" "10" but instead it is "5" "10". What could it mean?

(注意:根据用户 3667217 建议的编程指南编辑了代码)

推荐答案

提醒:当您使用多处理时，您需要在 'if __name__ == '__main__' 中启动一个进程:条款:(参见编程指南)

Reminder: when you're using multiprocessing, you need to start a process in an 'if __name__ == '__main__': clause: (see programming guidelines)

import pickle
import multiprocessing

class Tricky:
    def __init__(self,x):
        self.data=x

    def __setstate__(self, d):
        print('setstate happening')
        self.data = 10

    def __getstate__(self):
        return self.data
        print('getstate happening')

def report(ar,q):
    q.put(ar.data)

if __name__ == '__main__':
    ar = Tricky(5)
    q = multiprocessing.Queue()
    p = multiprocessing.Process(target=report, args=(ar, q))
    print('now starting process')
    p.start()
    print('now joining process')
    p.join()
    print('now getting results from queue')
    print(q.get())
    print('now getting pickle dumps')
    print(pickle.loads(pickle.dumps(ar)).data)   

在窗户上，我看到

now starting process
now joining process
setstate happening
now getting results from queue 
10
now getting pickle dumps
setstate happening
10

在 Ubuntu 上，我看到:

On Ubuntu, I see:

now starting process
now joining process
now getting results from queue
5
now getting pickle dumps
getstate happening
setstate happening
10

我想这应该可以回答您的问题.multiprocess 在 Windows 上调用 __setstate__ 方法，但在 Linux 上不调用.在 Linux 上，当你调用 pickle.dumps 时，它首先调用 __getstate__，然后是 __setstate__.有趣的是，多处理模块在不同平台上的表现如何不同.

I suppose this should answer your question. The multiprocess invokes __setstate__ method on Windows but not on Linux. And on Linux, when you call pickle.dumps it first call __getstate__, then __setstate__. It's interesting to see how multiprocessing module is behaving differently on different platforms.

这篇关于多处理忽略“__setstate__"；的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！