Python多处理和子进程的独立性 [英] Python multiprocessing and independence of children processes

问题描述

我从python终端运行以下命令，以生成长时间运行的子进程:

From the python terminal, I run some command like the following, to spawn a long-running child process:

from multiprocessing.process import Process
Process(target=LONG_RUNNING_FUNCTION).start()

此命令返回，并且我可以在python终端中执行其他操作，但是孩子打印的所有内容仍会打印到我的python终端会话中.

This command returns, and I can do other things in the python terminal, but anything printed by the child is still printed to my python terminal session.

当我退出终端(使用exit或 CTRL + D )时，它的退出命令将挂起.如果在此挂起期间按 CTRL + C ，子进程将终止.

When I exit the terminal (either with exit or CTRL+D), the exit command it hangs. If I hit CTRL+C during this hang, the child process is terminated.

如果我手动(通过posix kill命令)杀死了python终端进程，则子进程将被孤立，并继续运行，其输出可能会被丢弃.

If I kill the python terminal process manually (via the posix kill command), the child process is instead orphaned, and continues running with its output presumably discarded.

如果我使用python -c运行此代码，它将等待子级终止，并且 CTRL + C 会杀死父级和子级.

If I run this code with python -c, it waits for the child to terminate, and CTRL+C kills both parent and child.

当父母终止时，哪个运行的python配置会杀死孩子?特别是，如果python-mod_wsgi-apache网络服务器生成子进程然后重新启动，则这些子进程会被杀死吗?

Which run configurations of python kill children when the parents are terminated? In particular, if a python-mod_wsgi-apache webserver spawns child processes and then is restarted, are the children killed?

[顺便说一句，分离从终端产生的子进程的正确方法是什么?有没有一种比以下方法更优雅的方法:故意在python中创建一个孤立的进程]

[ As an aside, what is the proper way of detaching child processes spawned from the terminal? Is there a way more elegant than the following: Deliberately make an orphan process in python ]

更新:在重新启动apache时，multiprocessing.Process由在apache下运行的Web服务器生成的python子进程不会被杀死.

Update: python subprocesses spawned with multiprocessing.Process by a web server running under apache are not killed when apache is restarted.

推荐答案

这与您调用python的方式无关；这是multiprocessing模块的功能.导入该模块时，会将退出处理程序添加到父进程，该进程在允许父进程退出之前，对通过multiprocessing.Process创建的所有子进程的Process对象调用join().如果您打算以这种方式启动子进程，则没有办法，不对模块内部进行黑客操作，就可以避免给您带来麻烦的行为.

This isn't a matter of how you're invoking python; it's a feature of the multiprocessing module. When you import that module an exit handler is added to the parent process that calls join() on the Process objects of all children created via multiprocessing.Process before allowing the parent to exit. If you're going to start child processes in this fashion, there's no way, without hacking the module internals, to avoid the behavior that's giving you trouble.

如果您要启动一个进程，该进程的寿命可以超过父进程，那么使用subprocess.Popen可能会更好.如果孩子是这样开始的，那么父母不会在退出之前尝试加入孩子，而是会立即退出，留下一个孤儿:

If you want to start a process that's able to outlive the parent, then you'll probably be better served using subprocess.Popen. If the child's started that way, the parent won't attempt to join the child before exiting and will instead exit immediately, leaving an orphaned child:

>>> from subprocess import Popen
>>> Popen(["sleep", "100"])
<subprocess.Popen object at 0x10d3fedd0>
>>> exit()
alp:~ $ ps -opid,ppid,command | grep sleep | grep -v grep
37979     1 sleep 100

您使用multiprocessing而不是subprocess是否有特定原因?前者并非旨在用于创建旨在使父进程寿命更长的子进程.这是为了创建子进程来完成可以在多个CPU上并行进行的工作，这是规避全局解释器锁定的一种方式. (出于讨论的目的，我忽略了multiprocessing的分布式功能.)因此，multiprocessing通常用于那些没有GIL的情况下，您将使用线程的情况. (请注意，在这方面，multiprocessing模块的API紧随threading模块的API建模.)

Is there a particular reason you're using multiprocessing instead of subprocess? The former isn't intended to be used to create child processes meant to outlive the parent; it's for creating child processes to do work that can be usefully parallelized across CPUs, as a way of circumventing the Global Interpreter Lock. (I'm ignoring the distributed capabilities of multiprocessing for purposes of this discussion.) multiprocessing is thus normally used in those cases where, if there weren't a GIL, you would use threads. (Note, in this regard, that the multiprocessing module's API is modeled closely after that of the threading module.)

对帖子结尾处的具体问题:(1)当父级被终止时，关于python的一切都不负责杀死孩子.仅当父服务器在退出之前将其杀死(或者整个进程组都被杀死)时，该Web服务器的子服务器才会被杀死. (2)链接到的方法看起来像是在不知道标准惯用法的情况下尝试复制守护进程.有许多用于创建守护进程的软件包.您应该改用其中之一.

To the specific questions at the end of your post: (1) Nothing about python is responsible for killing children when the parent is terminated. A child of the web server will only be killed if the parent kills it before exiting (or if the entire process group is killed). (2) The method you link to looks like it's trying to replicate daemonization without knowledge of the standard idioms for doing that. There are a number of packages out there for creating daemon processes; you should use one of them instead.

这篇关于Python多处理和子进程的独立性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！