如何使用python的多处理程序终止进程 [英] how to to terminate process using python's multiprocessing

问题描述

我有一些代码需要在可能会挂起或无法控制的其他几个系统上运行.我想使用python的多处理程序来生成子进程以独立于主程序运行，然后在它们挂起或遇到问题时将其终止，但是我不确定最好的方法.

I have some code that needs to run against several other systems that may hang or have problems not under my control. I would like to use python's multiprocessing to spawn child processes to run independent of the main program and then when they hang or have problems terminate them, but I am not sure of the best way to go about this.

当调用终止时，它确实杀死了子进程，但随后它变成了一个已消失的僵尸，直到进程对象消失后才释放它.循环永无休止的示例代码可以杀死它，并在再次调用时允许重新生成，但是这似乎不是解决此问题的好方法(例如，在__init __()中，multiprocessing.Process()会更好).

When terminate is called it does kill the child process, but then it becomes a defunct zombie that is not released until the process object is gone. The example code below where the loop never ends works to kill it and allow a respawn when called again, but does not seem like a good way of going about this (ie multiprocessing.Process() would be better in the __init__()).

有人有建议吗?

class Process(object):
    def __init__(self):
        self.thing = Thing()
        self.running_flag = multiprocessing.Value("i", 1)

    def run(self):
        self.process = multiprocessing.Process(target=self.thing.worker, args=(self.running_flag,))
        self.process.start()
        print self.process.pid

    def pause_resume(self):
        self.running_flag.value = not self.running_flag.value

    def terminate(self):
        self.process.terminate()

class Thing(object):
    def __init__(self):
        self.count = 1

    def worker(self,running_flag):
        while True:
            if running_flag.value:
                self.do_work()

    def do_work(self):
        print "working {0} ...".format(self.count)
        self.count += 1
        time.sleep(1)

推荐答案

您可以在后台将子进程作为守护程序运行.

You might run the child processes as daemons in the background.

process.daemon = True

守护进程中的任何错误和挂起(或无限循环)都不会影响主进程，并且只有在主进程退出后才会终止.

Any errors and hangs (or an infinite loop) in a daemon process will not affect the main process, and it will only be terminated once the main process exits.

这将适用于简单的问题，直到您遇到许多子守护程序时为止，这些子守护程序将在没有任何显式控制的情况下不断从父进程中获取内存.

This will work for simple problems until you run into a lot of child daemon processes which will keep reaping memories from the parent process without any explicit control.

最好的方法是设置一个Queue以使所有子进程与父进程进行通信，以便我们可以join对其进行清理.以下是一些简单的代码，这些代码将检查子处理是否挂起(也称为time.sleep(1000))，并将消息发送到队列，以使主进程对其执行操作:

Best way is to set up a Queue to have all the child processes communicate to the parent process so that we can join them and clean up nicely. Here is some simple code that will check if a child processing is hanging (aka time.sleep(1000)), and send a message to the queue for the main process to take action on it:

import multiprocessing as mp
import time
import queue

running_flag = mp.Value("i", 1)

def worker(running_flag, q):
    count = 1
    while True:
        if running_flag.value:
            print "working {0} ...".format(count)
            count += 1
            q.put(count)
            time.sleep(1)
            if count > 3:
                # Simulate hanging with sleep
                print "hanging..."
                time.sleep(1000)

def watchdog(q):
    """
    This check the queue for updates and send a signal to it
    when the child process isn't sending anything for too long
    """
    while True:
        try:
            msg = q.get(timeout=10.0)
        except queue.Empty as e:
            print "[WATCHDOG]: Maybe WORKER is slacking"
            q.put("KILL WORKER")

def main():
    """The main process"""
    q = mp.Queue()

    workr = mp.Process(target=worker, args=(running_flag, q))
    wdog = mp.Process(target=watchdog, args=(q,))

    # run the watchdog as daemon so it terminates with the main process
    wdog.daemon = True

    workr.start()
    print "[MAIN]: starting process P1"
    wdog.start()

    # Poll the queue
    while True:
        msg = q.get()
        if msg == "KILL WATCHDOG":
            print "[MAIN]: Terminating slacking WORKER"
            workr.terminate()
            time.sleep(0.1)
            if not workr.is_alive():
                print "[MAIN]: WORKER is a goner"
                workr.join(timeout=1.0)
                print "[MAIN]: Joined WORKER successfully!"
                q.close()
                break # watchdog process daemon gets terminated

if __name__ == '__main__':
    main()

在不终止worker的情况下，由于worker从未完成，因此尝试将其join()锁定到主进程将永远受阻.

Without terminating worker, attempt to join() it to the main process would have blocked forever since worker has never finished.

这篇关于如何使用python的多处理程序终止进程的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！