导入和使用在不导致Windows无限循环的情况下使用多处理的模块 [英] importing and using a module that uses multiprocessing without causing infinite loop on Windows
问题描述
我有一个名为multi.py
的模块.如果我只是想将multi.py作为脚本执行,那么避免在Windows上崩溃(产生无限数量的进程)的解决方法是将多重处理代码放在以下位置:
I have a module named multi.py
. If I simply wanted to execute multi.py as a script, then the workaround to avoid crashing on Windows (spawning an infinite number of processes) is to put the multiprocessing code under:
if __name__ == '__main__':
但是,我试图将其作为模块从另一个脚本导入并调用multi.start()
.如何做到这一点?
However, I am trying to import it as a module from another script and call multi.start()
. How can this be accomplished?
# multi.py
import multiprocessing
def test(x):
x**=2
def start():
pool = multiprocessing.Pool(processes=multiprocessing.cpu_count()-2)
pool.map(test, (i for i in range(1000*1000)))
pool.terminate()
print('done.')
if __name__ == '__main__':
print('runs as a script,',__name__)
else:
print('runs as imported module,',__name__)
这是我运行的test.py
:
# test.py
import multi
multi.start()
推荐答案
我不太了解您的要求.您无需执行任何操作即可阻止此操作产生无限多个进程.我只是在Windows XP上运行它---导入了文件并运行了multi.start()
---它在几秒钟内就完成了.
I don't quite get what you're asking. You don't need to do anything to prevent this from spawning infinitely many processes. I just ran it on Windows XP --- imported the file and ran multi.start()
--- and it completed fine in a couple seconds.
必须执行if __name__=="__main__"
保护的原因是,在Windows上,多处理必须导入主脚本才能运行目标函数,这意味着将执行该文件中的顶级模块代码.仅当该顶级模块代码本身尝试生成新进程时,才会出现该问题.在您的示例中,顶层模块代码不使用多处理,因此没有无限的过程链.
The reason you have to do the if __name__=="__main__"
protection is that, on Windows, multiprocessing has to import the main script in order to run the target function, which means top-level module code in that file will be executed. The problem only arises if that top-level module code itself tries to spawn a new process. In your example, the top level module code doesn't use multiprocessing, so there's no infinite process chain.
现在,我明白了您的要求.您不需要保护multi.py
.您需要保护您的主脚本,无论它是什么.如果您崩溃了,那是因为在您的主脚本中,您正在顶级模块代码中执行multi.start()
.您的脚本需要看起来像这样:
Now I get what you're asking. You don't need to protect multi.py
. You need to protect your main script, whatever it is. If you're getting a crash, it's because in your main script you are doing multi.start()
in the top level module code. Your script needs to look like this:
import multi
if __name__=="__main__":
multi.start()
主要脚本中始终需要保护".
The "protection" is always needed in the main script.
这篇关于导入和使用在不导致Windows无限循环的情况下使用多处理的模块的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!