在 Windows 上导入和使用使用多处理而不会导致无限循环的模块 [英] importing and using a module that uses multiprocessing without causing infinite loop on Windows

问题描述

我有一个名为 multi.py 的模块.如果我只是想将 multi.py 作为脚本执行，那么避免在 Windows 上崩溃(产生无限数量的进程)的解决方法是将多处理代码放在:

I have a module named multi.py. If I simply wanted to execute multi.py as a script, then the workaround to avoid crashing on Windows (spawning an infinite number of processes) is to put the multiprocessing code under:

if __name__ == '__main__':

但是，我尝试将它作为模块从另一个脚本导入并调用 multi.start().这如何实现?

However, I am trying to import it as a module from another script and call multi.start(). How can this be accomplished?

# multi.py
import multiprocessing

def test(x):
    x**=2

def start():
    pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-2)
    pool.map(test, (i for i in range(1000*1000)))
    pool.terminate()
    print('done.')

if __name__ == '__main__':
    print('runs as a script,',__name__)
else:
    print('runs as imported module,',__name__)

这是我运行的 test.py:

# test.py
import multi
multi.start()

推荐答案

我不太明白你在问什么.你不需要做任何事情来防止它产生无限多的进程.我只是在 Windows XP 上运行它 --- 导入文件并运行 multi.start() --- 它在几秒钟内完成.

I don't quite get what you're asking. You don't need to do anything to prevent this from spawning infinitely many processes. I just ran it on Windows XP --- imported the file and ran multi.start() --- and it completed fine in a couple seconds.

您必须执行 if __name__=="__main__" 保护的原因是，在 Windows 上，多处理必须导入主脚本才能运行目标函数，这意味着顶部-该文件中的级别模块代码将被执行.仅当顶级模块代码本身尝试生成新进程时，才会出现问题.在您的示例中，顶级模块代码不使用多处理，因此没有无限的进程链.

The reason you have to do the if __name__=="__main__" protection is that, on Windows, multiprocessing has to import the main script in order to run the target function, which means top-level module code in that file will be executed. The problem only arises if that top-level module code itself tries to spawn a new process. In your example, the top level module code doesn't use multiprocessing, so there's no infinite process chain.

现在我明白你的要求了.您不需要保护 multi.py.你需要保护你的主脚本，不管它是什么.如果您遇到崩溃，那是因为在您的主脚本中，您在顶级模块代码中执行 multi.start().您的脚本需要如下所示:

Now I get what you're asking. You don't need to protect multi.py. You need to protect your main script, whatever it is. If you're getting a crash, it's because in your main script you are doing multi.start() in the top level module code. Your script needs to look like this:

import multi
if __name__=="__main__":
    multi.start()

main 脚本中始终需要保护".

The "protection" is always needed in the main script.

这篇关于在 Windows 上导入和使用使用多处理而不会导致无限循环的模块的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！