分析多进程Python脚本时出现神秘的泡菜错误 [英] Mysterious pickle error while profiling a multi-process Python script

问题描述

我使用的是multiprocessing模块，我使用的是通过multiprocessing.Queue对象发送的UpdateMessage对象(我自己的类)，以在进程之间进行通信.这是类:

I"m using the multiprocessing module, and I'm using UpdateMessage objects (my own class), sent through multiprocessing.Queue objects, to communicate between processes. Here's the class:

class UpdateMessage:
    def __init__(self, arrayref, rowslice, colslice, newval):
        self.arrayref = arrayref
        self.rowslice = rowslice
        self.colslice = colslice
        self.newval = newval
    def do_update(self):
        if self.arrayref == 'uL':
            arr = uL
        elif self.arrayref == 'uR':
            arr = uR
        else:
            raise Exception('UpdateMessage.arrayref neither uL nor uR')
        arr[self.rowslice, self.colslice] = self.newval

当我运行脚本时，它运行良好.但是，当我用cProfile或profile运行它时，会出现以下错误:

When I run the script, it works perfectly fine. However, when I run it with either cProfile or profile, it gives the following error:

_pickle.PicklingError: Can't pickle <class '__main__.UpdateMessage'>: attribute lookup __main__.UpdateMessage failed

这似乎是在使班级泡菜，但我不明白为什么会发生这种情况.我的代码没有执行此操作，没有它也可以正常工作，因此它可能是multiprocessing模块.但是为什么需要腌制UpdateMessage，我该如何解决错误?

It seems to be trying to pickle the class, but I can't see why this is happening. My code doesn't do this, and it works just fine without it, so it's probably the multiprocessing module. But why would it need to pickle UpdateMessage, and how can I fix the error?

这是发送UpdateMessage的代码的一部分(脚本的多个部分都执行此操作，但是全部以相同的方式进行):

here's part of the code that's sending the UpdateMessage (multiple parts of the script do this, but all in the same way):

msg = UpdateMessage(uLref, refer[0] + marker[0] - 2,
                    slice(uL.shape[1]), ustar.copy())
queue.put(msg)

回溯不是很有帮助:

Traceback (most recent call last):
  File "/usr/lib/python3.2/multiprocessing/queues.py", line 272, in _feed
    send(obj)

推荐答案

我遇到了同样的问题，并通过在自己的文件中定义要腌制的类来解决了这个问题.

I had the same problem, and solved it by defining the class to be pickled in its own file.

这篇关于分析多进程Python脚本时出现神秘的泡菜错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！