Python多处理-捕获信号以重新启动子进程或关闭父进程 [英] Python multiprocessing - Capturing signals to restart child processes or shut down parent process

问题描述

我正在使用多处理库生成两个子进程.我想确保只要父进程还活着，如果子进程死了(收到SIGKILL或SIGTERM)，它们将自动重启.另一方面，如果父进程收到SIGTERM/SIGINT，我希望它终止所有子进程然后退出.

I am using the multiprocessing library to spawn two child processes. I would like to ensure that as long as the parent process is alive, if the child processes die (receive a SIGKILL or SIGTERM), that they are restarted automatically. On the other hand, if the parent process receives a SIGTERM/SIGINT, I want it to terminate all child processes then exit.

这是我解决问题的方式:

This is how I approached the problem:

import sys
import time
from signal import signal, SIGINT, SIGTERM, SIGQUIT, SIGCHLD, SIG_IGN
from functools import partial
import multiprocessing
import setproctitle

class HelloWorld(multiprocessing.Process):
    def __init__(self):
        super(HelloWorld, self).__init__()

        # ignore, let parent handle it
        signal(SIGTERM, SIG_IGN)

    def run(self):

        setproctitle.setproctitle("helloProcess")

        while True:
            print "Hello World"
            time.sleep(1)

class Counter(multiprocessing.Process):
    def __init__(self):
        super(Counter, self).__init__()

        self.counter = 1

        # ignore, let parent handle it
        signal(SIGTERM, SIG_IGN)

    def run(self):

        setproctitle.setproctitle("counterProcess")

        while True:
            print self.counter
            time.sleep(1)
            self.counter += 1


def signal_handler(helloProcess, counterProcess, signum, frame):

    print multiprocessing.active_children()
    print "helloProcess: ", helloProcess
    print "counterProcess: ", counterProcess

    if signum == 17:

        print "helloProcess: ", helloProcess.is_alive()

        if not helloProcess.is_alive():
            print "Restarting helloProcess"

            helloProcess = HelloWorld()
            helloProcess.start()

        print "counterProcess: ", counterProcess.is_alive()

        if not counterProcess.is_alive():
            print "Restarting counterProcess"

            counterProcess = Counter()
            counterProcess.start()

    else:

        if helloProcess.is_alive():
            print "Stopping helloProcess"
            helloProcess.terminate()

        if counterProcess.is_alive():
            print "Stopping counterProcess"
            counterProcess.terminate()

        sys.exit(0)



if __name__ == '__main__':

    helloProcess = HelloWorld()
    helloProcess.start()

    counterProcess = Counter()
    counterProcess.start()

    for signame in [SIGINT, SIGTERM, SIGQUIT, SIGCHLD]:
        signal(signame, partial(signal_handler, helloProcess, counterProcess))

    multiprocessing.active_children()

如果我将SIGKILL发送给counterProcess，它将正确重启.但是，将SIGKILL发送到helloProcess还会重新启动counterProcess而不是helloProcess吗?

If I send a SIGKILL to the counterProcess, it will restart correctly. However, sending a SIGKILL to the helloProcess also restarts the counterProcess instead of the helloProcess?

如果我将SIGTERM发送到父进程，则父进程将退出，但子进程将变为孤立进程并继续运行.我该如何纠正这种行为?

If I send a SIGTERM to the parent process, the parent will exit, but the child processes become orphans and continue on. How do I correct this behavior?

推荐答案

要从 signal.SIGCHLD 处理程序中重新创建死亡的孩子，母亲必须调用 os.wait 之一功能，因为 Process.is_alive 在这里不起作用.
虽然可能，但很复杂，因为 signal.SIGCHLD 会在其子状态之一更改为f时交付给母亲.子级收到 signal.SIGSTOP ， signal.SIGCONT 或任何其他终止信号.
因此， signal.SIGCHLD 处理程序必须区分孩子的这些状态.仅在交付 signal.SIGCHLD 时重新创建子代可能会创建过多的子代.

To recreate dead children from signal.SIGCHLD handler, the mother must call one of os.wait functions, because Process.is_alive doesn't work here.
Though possible, it's complicated, because signal.SIGCHLD is delivered to mother when one of it's children status changes f.e. signal.SIGSTOP, signal.SIGCONT or any other terminating signals are received by the child.
So the signal.SIGCHLD handler must differentiate between theses states of the child. Just merely recreating children when signal.SIGCHLD delivered may create more children than necessary.

以下代码将 os.waitpid 与 os.WNOHANG 结合使用，使其变为非阻塞状态，并分别将 os.WUNTRACED 和 os.WCONTINUED ，用于了解 signal.SIGCHLD 是来自 signal.SIGSTOP 还是 signal.SIGCONT .
os.waitpid 不起作用，即如果任何 Process 实例是 print (0，0)> ed，即 str(Process())，然后再调用 os.waitpid .

The following code uses os.waitpid with os.WNOHANG to make it non-blocking and os.WUNTRACED and os.WCONTINUED for learning if signal.SIGCHLD is from signal.SIGSTOP or signal.SIGCONT.
os.waitpid doesn't work, i.e. returns (0, 0) if any of the Process instance is printed, i.e str(Process()) before you call os.waitpid.

import sys
import time
from signal import signal, pause, SIGINT, SIGTERM, SIGQUIT, SIGCHLD, SIG_DFL
import multiprocessing
import os

class HelloWorld(multiprocessing.Process):
    def run(self):
        # reset SIGTERM to default for Process.terminate to work
        signal(SIGTERM, SIG_DFL)
        while True:
            print "Hello World"
            time.sleep(1)

class Counter(multiprocessing.Process):
    def __init__(self):
        super(Counter, self).__init__()
        self.counter = 1

    def run(self):
        # reset SIGTERM to default for Process.terminate to work
        signal(SIGTERM, SIG_DFL)
        while True:
            print self.counter
            time.sleep(1)
            self.counter += 1


def signal_handler(signum, _):
    global helloProcess, counterProcess

    if signum == SIGCHLD:
        pid, status = os.waitpid(-1, os.WNOHANG|os.WUNTRACED|os.WCONTINUED)
        if os.WIFCONTINUED(status) or os.WIFSTOPPED(status):
            return
        if os.WIFSIGNALED(status) or os.WIFEXITED(status):
            if helloProcess.pid == pid:
                print("Restarting helloProcess")
                helloProcess = HelloWorld()
                helloProcess.start()

            elif counterProcess.pid == pid:
                print("Restarting counterProcess")
                counterProcess = Counter()
                counterProcess.start()

    else:
        # mother shouldn't be notified when it terminates children
        signal(SIGCHLD, SIG_DFL)
        if helloProcess.is_alive():
            print("Stopping helloProcess")
            helloProcess.terminate()

        if counterProcess.is_alive():
            print("Stopping counterProcess")
            counterProcess.terminate()

        sys.exit(0)

if __name__ == '__main__':

    helloProcess = HelloWorld()
    helloProcess.start()

    counterProcess = Counter()
    counterProcess.start()

    for signame in [SIGINT, SIGTERM, SIGQUIT, SIGCHLD]:
        signal(signame, signal_handler)

    while True:
        pause()

以下代码无需使用 signal.SIGCHLD 即可重新创建死亡的子代.因此，它比前一个简单.
创建两个孩子之后，母进程为SIGINT，SIGTERM和SIGQUIT设置一个名为 term_child 的信号处理程序. term_child 在调用时终止并加入每个子级.

The following code recreates dead children without using signal.SIGCHLD. So it's simpler than the former.
Having created two children, mother process sets a signal handler named term_child for SIGINT, SIGTERM, SIGQUIT. term_child terminates and joins each child upon invocation.

母亲进程会继续检查孩子是否还活着，并在必要时在 while 循环中重新创建它们.

The mother process keeps checking if children are alive, and recreates them if necessary in the while loop.

由于每个孩子都从母亲那里继承了信号处理程序，因此应将 SIGINT 处理程序重置为其默认值，以使 Process.terminate 正常工作

Because every child inherits signal handlers from mother, the SIGINT handler should be reset to its default value for Process.terminate to work

import sys
import time
from signal import signal, SIGINT, SIGTERM, SIGQUIT
import multiprocessing

class HelloWorld(multiprocessing.Process):    
    def run(self):
        signal(SIGTERM, SIG_DFL)
        while True:
            print "Hello World"
            time.sleep(1)

class Counter(multiprocessing.Process):
    def __init__(self):
        super(Counter, self).__init__()
        self.counter = 1

    def run(self):
        signal(SIGTERM, SIG_DFL)
        while True:
            print self.counter
            time.sleep(1)
            self.counter += 1

def term_child(_, __):
    for child in children:
        child.terminate()
        child.join()
    sys.exit(0)

if __name__ == '__main__':

    children = [HelloWorld(), Counter()]
    for child in children:
        child.start()

    for signame in (SIGINT, SIGTERM, SIGQUIT):
        signal(signame, term_child)

    while True:
        for i, child in enumerate(children):
            if not child.is_alive():
                children[i] = type(child)()
                children[i].start()
        time.sleep(1)

这篇关于Python多处理-捕获信号以重新启动子进程或关闭父进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！