Python：从多处理进程中获取回溯 [英] Python: Getting a traceback from a multiprocessing.Process

问题描述

我正在尝试从多处理进程中获取跟踪对象。
不幸地通过管道传递异常信息不起作用，因为追溯对象不能被腌制：

I am trying to get hold of a traceback object from a multiprocessing.Process. Unfortunately passing the exception info through a pipe does not work because traceback objects can not be pickled:

def foo(pipe_to_parent):
    try:
        raise Exception('xxx')
    except:
        pipe_to_parent.send(sys.exc_info())

to_child, to_self = multiprocessing.Pipe()
process = multiprocessing.Process(target = foo, args = (to_self,))
process.start()
exc_info = to_child.recv()
process.join()
print traceback.format_exception(*exc_info)
to_child.close()
to_self.close()

追溯：

Traceback (most recent call last):
  File "/usr/lib/python2.6/multiprocessing/process.py", line 231, in _bootstrap
    self.run()
  File "/usr/lib/python2.6/multiprocessing/process.py", line 88, in run
    self._target(*self._args, **self._kwargs)
  File "foo", line 7, in foo
    to_parent.send(sys.exc_info())
PicklingError: Can't pickle <type 'traceback'>: attribute lookup __builtin__.traceback failed

还有另外一种方式来访问异常信息？我想避免传递格式化的字符串。

Is there another way to access the exception info? I'd like to avoid passing the formatted string.

推荐答案

使用 tblib 你可以传递包装的例外并稍后重新提出：

Using tblib you can pass wrapped exceptions and reraise them later:

import tblib.pickling_support
tblib.pickling_support.install()

from multiprocessing import Pool
import sys


class ExceptionWrapper(object):

    def __init__(self, ee):
        self.ee = ee
        __,  __, self.tb = sys.exc_info()

    def re_raise(self):
        raise self.ee.with_traceback(self.tb)
        # for Python 2 replace the previous line by:
        # raise self.ee, None, self.tb


# example how to use ExceptionWrapper

def inverse(i):
    """will fail for i == 0"""
    try:
        return 1.0 / i
    except Exception as e:
        return ExceptionWrapper(e)


def main():
    p = Pool(1)
    results = p.map(inverse, [0, 1, 2, 3])
    for result in results:
        if isinstance(result, ExceptionWrapper):
            result.re_raise()


if __name__ == "__main__":
    main()

所以，如果你在远程进程中遇到异常，请用 ExceptionWrapper 然后将其传回。在主流程中调用 re_reraise 可以完成工作。

So, if you catch an exception in your remote process, wrap it with ExceptionWrapper and then pass it back. Calling re_reraise in the main process will do the work.

这篇关于Python：从多处理进程中获取回溯的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！