多处理启动了太多的Python VM实例 [英] Multiprocessing launching too many instances of Python VM

问题描述

我正在编写一些多处理代码(Python 2.6.4，WinXP)，这些代码会生成运行后台任务的进程.在处理一些琐碎的示例时，我遇到了一个问题，即即使我只告诉它产生一个固定的数字，我的代码也不断产生新的进程.

I am writing some multiprocessing code (Python 2.6.4, WinXP) that spawns processes to run background tasks. In playing around with some trivial examples, I am running into an issue where my code just continuously spawns new processes, even though I only tell it to spawn a fixed number.

程序本身运行良好，但是如果我在Windows TaskManager中查看，则会不断看到新的"python.exe"进程出现.它们只是在程序运行时不断产生(最终使我的机器饿死).

The program itself runs fine, but if I look in Windows TaskManager, I keep seeing new 'python.exe' processes appear. They just keep spawning more and more as the program runs (eventually starving my machine).

例如，
我希望下面的代码能够启动2个python.exe进程.第一个是程序本身，第二个是它产生的子进程.知道我在做什么错吗?

For example,
I would expect the code below to launch 2 python.exe processes. The first being the program itself, and the second being the child process it spawns. Any idea what I am doing wrong?

import time
import multiprocessing


class Agent(multiprocessing.Process):
    def __init__(self, i):
        multiprocessing.Process.__init__(self)
        self.i = i

    def run(self):
        while True:
            print 'hello from %i' % self.i
            time.sleep(1)


agent = Agent(1)
agent.start()

推荐答案

您似乎没有认真遵循文档中的指南，特别是

It looks like you didn't carefully follow the guidelines in the documentation, specifically this section where it talks about "Safe importing of main module".

我相信，您需要使用if __name__ == '__main__':块来保护启动代码，否则您将得到所得到的.

You need to protect your launch code with an if __name__ == '__main__': block or you'll get what you're getting, I believe.

我认为，归结为多处理模块不能像在Linux上那样使用os.fork()，在Linux上，已经在运行的进程基本上被克隆到内存中.在Windows(没有此类fork())上，它必须运行一个新的Python解释器，并告诉它导入您的主模块，然后在完成后执行start/run方法.如果您有模块级别"的代码，没有名称检查的保护，那么在导入过程中，它将重新开始整个序列，无限制地

I believe it comes down to the multiprocessing module not being able to use os.fork() as it does on Linux, where an already-running process is basically cloned in memory. On Windows (which has no such fork()) it must run a new Python interpreter and tell it to import your main module and then execute the start/run method once that's done. If you have code at "module level", unprotected by the name check, then during the import it starts the whole sequence over again, ad infinitum

这篇关于多处理启动了太多的Python VM实例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！