Windows 上的 Python 多处理运行时错误 [英] Python Multiprocessing RuntimeError on Windows

问题描述

我有一个类函数(我们称之为alpha.py")，它使用多处理(processes=2)来派生一个进程，并且是我编写的 Python 包的一部分.在一个单独的 Python 脚本(我们称之为beta.py")中，我从这个类中实例化了一个对象，并调用了使用多处理的相应函数.最后，所有这些都被包装在一个处理许多不同类对象和函数的包装 Python 脚本(我们称之为gamma.py")中.

I have a class function (let's call it "alpha.py") that uses multiprocessing (processes=2) to fork a process and is part of a Python package that I wrote. In a separate Python script (let's call it "beta.py"), I instantiated an object from this class and called the corresponding function that uses multiprocessing. Finally, all of this is wrapped inside a wrapper Python script (let's call this "gamma.py") that handles many different class objects and functions.

本质上:

1. 从命令行运行 ./gamma.py
2. gamma.py 使用子进程并执行 beta.py
3. beta.py 从 alpha.py 类实例化一个对象并调用使用多处理 (processes=2) 的函数

这在 Mac 或 Linux 上运行没有问题.然而，它在 Windows 机器上成为一个问题，错误(和文档)表明我应该在某处写这个:

This has no problems being run on a Mac or Linux. However, it becomes a problem on a Windows machine and the error (and documentation) suggests that I should write this somewhere:

if __name__ == '__main__':
    freeze_support()

另一篇文章也提到了做同样的事情.

This other post also mentions doing the same thing.

但是，我不知道这两行的确切位置.目前，alpha.py、beta.py 或 gamma.py 都不包含 if __name__ == '__main__': 部分.如果有人能告诉我这两行应该去哪里以及它背后的基本原理，那就太好了.

However, I don't know exactly where these two lines should reside. Currently, neither alpha.py, beta.py, or gamma.py contains an if __name__ == '__main__': section. It would be great if somebody can tell me where these two lines should go and also the rationale behind it.

推荐答案

实际上，这里不需要 freeze_support().您会收到 RuntimeError，因为您在 beta 模块的顶层创建并启动了新进程.

Actually, freeze_support() is not needed here. You get a RuntimeError because you create and start your new processes at the top level of your beta module.

在 Windows 上使用 multiprocessing 创建新进程时，将在此进程中启动一个新的 Python 解释器，并尝试导入具有应执行的目标函数的模块.这是您的 beta 模块.现在，当您导入它时，应该执行所有顶级语句，这将导致创建并再次启动新进程.然后，递归地，从那个过程中产生另一个过程，依此类推.

When a new process is created using multiprocessing on Windows, a new Python interpreter will be started in this process and it will try to import the module with the target function that should be executed. This is your beta module. Now, when you import it, all your top level statements should be executed which will cause a new process to be created and started again. And then, recursively, another process from that process, and so on and so forth.

这很可能不是您想要的，因此当您使用 subprocess 直接运行 beta.py 时，新进程应该只初始化和启动一次.

This is most likely not what you want, thus new processes should be initialized and started only once, when you run beta.py directly with a subprocess.

if __name__ == '__main__': 应该放在 beta.py 中，然后在本节中移动新进程的初始化和启动代码.之后，当 beta.py 将被导入而不直接运行时，将不会启动任何新进程，您将不会看到任何副作用.

if __name__ == '__main__': should be placed in beta.py, then move initialization and start code for your new processes in this section. After that, when beta.py will be imported and not run directly, no new process will be started and you will not see any side effects.

这篇关于Windows 上的 Python 多处理运行时错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！