Python多处理退出错误 [英] Python Multiprocessing exit error

问题描述

当我按Ctrl-C退出我的应用程序时，我会看到此消息

I am seeing this when I press Ctrl-C to exit my app

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.6/multiprocessing/util.py", line 269, in _exit_function
    p.join()
  File "/usr/lib/python2.6/multiprocessing/process.py", line 119, in join
    res = self._popen.wait(timeout)
  File "/usr/lib/python2.6/multiprocessing/forking.py", line 117, in wait
    return self.poll(0)
  File "/usr/lib/python2.6/multiprocessing/forking.py", line 106, in poll
    pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/usr/lib/python2.6/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/usr/lib/python2.6/multiprocessing/util.py", line 269, in _exit_function
    p.join()
  File "/usr/lib/python2.6/multiprocessing/process.py", line 119, in join
    res = self._popen.wait(timeout)
  File "/usr/lib/python2.6/multiprocessing/forking.py", line 117, in wait
    return self.poll(0)
  File "/usr/lib/python2.6/multiprocessing/forking.py", line 106, in poll
    pid, sts = os.waitpid(self.pid, flag)
OSError: [Errno 4] Interrupted system call

我在自己的东西上使用扭曲的东西，

I am using twisted on top of my own stuff,

我用以下代码注册了Ctrl-C信号

I registered the signal Ctrl-C with the following code

    def sigHandler(self, arg1, arg2):
        if not self.backuped:
            self.stopAll()
        else:
            out('central', 'backuped ALREADY, now FORCE exiting')
            exit()


    def stopAll(self):
        self.parserM.shutdown()
        for each in self.crawlM:
            each.shutdown()
        self.backup()
        reactor.stop()

，并且当他们向其他人发出要关机的信号时，它会试图告诉他们正确关机 通过

and when they signal others to shutdown, it tries to tell them to shutdown nicely through

exit = multiprocessing.Event()
def shutdown(self):
    self.exit.set()

我所有的流程都以某种形式存在，

where all my processes are in some form,

def run(self):
    while not self.exit.is_set():
        do something
    out('crawler', 'crawler exited sucessfully')

知道这个错误是什么吗?我只有在一个特定线程有多个实例时才知道它.

Any idea what this error is? I only get it when I have more than one instance of a particular thread.

推荐答案

这与OS系统调用，信号及其在多处理模块中的交互方式有关.我不太确定这是错误还是功能，但由于python与os相遇，因此它处于棘手的领域.

This is related to interactions OS system calls, signals and how it's handled in the multiprocessing module. I'm not really sure if it's a bug or a feature, but it's in somewhat tricky territory as it's where python meets the os.

问题在于，多处理在waitpid上阻塞，直到它正在等待的子项终止为止.但是，由于您已经为SIGINT安装了信号处理程序，并且您的程序获取了该信号，因此它会中断系统调用以执行您的信号处理程序，并且waitpid退出，表明它已被信号中断. python处理这种情况的方式是例外.

The problem is that multiprocessing is blocking on waitpid until the child it's waiting for has terminated. However, since you've installed a signal-handler for SIGINT and your program gets this signal, it interrupts the system call to execute your signal handler, and waitpid exits indicating that it was interrupted by a signal. The way python handles this case is by exceptions.

作为一种解决方法，您可以将有问题的部分包含在while循环中，并在等待线程完成的地方或像这样的多处理子类中尝试/捕获这样的块.Popen:

As a workaround, you can enclose the offending section(s) in a while-loop and try/catch blocks like this, either around where you wait for threads to finish, or subclass multiprocessing.Popen:

import errno
from multiprocessing import Process

p = Process( target=func, args=stuff )
p.start()
notintr = False
while not notintr:
  try:
    p.join() # "Offending code"
    notintr = True
  except OSError, ose:
    if ose.errno != errno.EINTR:
      raise ose

要想搞混multiprocessing.forking.Popen，您必须执行以下操作:

For mucking about with multiprocessing.forking.Popen you'd have to do something like this:

import errno
from multiprocessing import Process
from multiprocessing.forking import Popen
import os

# see /path/to/python/libs/multiprocessing/forking.py
class MyPopen(Popen):
  def poll(self, flag=os.WNOHANG): # from forking.py
    if self.returncode is None: # from forking.py
      notintr = False
      while not notintr:
        try:
          pid, sts = os.waitpid(self.pid, flag) # from forking.py
          notintr = True
        except OSError, ose:
          if ose.errno != errno.EINTR:
            raise ose
      # Rest of Popen.poll from forking.py goes here

p = Process( target=func args=stuff )
p._Popen = p
p.start()
p.join()

这篇关于Python多处理退出错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！