在python多处理中查找异常 [英] Finding exception in python multiprocessing

问题描述

我有一些类似以下的python代码:

I have a bit of python code that looks like this:

procs = cpu_count()-1
if serial or procs == 1:
    results = map(do_experiment, experiments)
else:
    pool = Pool(processes=procs)     
    results = pool.map(do_experiment, experiments)

当我设置serial标志时，它运行良好，但是当使用Pool时，它给出以下错误.当我尝试从do_experiment打印某些内容时，没有任何显示，因此我无法尝试/捕获并打印堆栈跟踪.

It runs fine when I set the serial flag, but it gives the following error when the Pool is used. When I try to print something from do_experiment nothing shows up, so I can't try/catch there and print a stack trace.

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 530, in __bootstrap_inner
    self.run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 483, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/multiprocessing/pool.py", line 285, in _handle_tasks
    put(task)
TypeError: 'NoneType' object is not callable

什么是进行调试的好方法?

What is a good way to proceed debugging this?

推荐答案

我回到了git历史，直到找到一个仍然可以正常工作的提交.

I went back in my git history until I found a commit where things were still working.

我在我的代码中添加了一个扩展dict的类，以便可以使用.访问键(因此，使用dict.foo代替dict["foo"].使用普通指令，多处理对此并不友好.解决了这个问题.

I added a class to my code that extends dict so that keys can be accessed with a . (so dict.foo in stead of dict["foo"]. Multiprocessing did not take kindly to this, using an ordinary dict solved the problem.

这篇关于在python多处理中查找异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！