Python Multiprocessing:并行执行前后串行执行代码 [英] Python Multiprocessing: Execute code serially before and after parallel execution

问题描述

这里的新手:我正在尝试串行执行一些代码，然后创建一个线程池并并行执行一些代码.并行执行完成后，我想串行执行更多代码.

Novice here: I am trying to execute some code serially and then create a pool of threads and execute some code in parallel. After the parallel execution is done, I want to execute some more code serially.

例如...

import time
from multiprocessing import Pool

print("I only want to print this statement once")



def worker(i):
    """worker function"""
    now = time.time()
    time.sleep(i)
    then = time.time()
    print(now, then)

if __name__ == '__main__':
    with Pool(3) as p:
        p.map(worker, [1, 1, 1])
        p.close()

print("Only print this once as well")

我希望这个返回...

I only want to print this statement once
1533511478.0619314 1533511479.0620182
1533511478.0789354 1533511479.0791905
1533511478.0979397 1533511479.098235
Only print this once as well

然而它返回的是:

I only want to print this statement once
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
I only want to print this statement once
Only print this once as well
1533511478.0619314 1533511479.0620182
1533511478.0789354 1533511479.0791905
1533511478.0979397 1533511479.098235
Only print this once as well

所以它似乎为每个池额外运行打印语句.

So it seems to be running the print statements an additional time for each pool.

任何帮助将不胜感激！

推荐答案

根据观察到的行为，我假设您使用的是 NT/Windows 操作系统.

Based on the observed behaviour, I assume you are on a NT/Windows Operating System.

您看到所有这些打印的原因是因为在 Windows 上 spawn 启动策略.当产生"一个新进程时，会启动一个新的 Python 解释器并接收模块和它应该执行的函数.当新的解释器导入模块时，会执行顶级的 print 函数.因此，重复打印.

The reason you see all those prints is because on Windows the spawn start strategy is used. When a new process is "spawned", a new Python interpreter is launched and it receives the module and the function it's supposed to execute. When the new interpreter imports the module, the top level print functions are executed. Hence the duplicate prints.

只需将那些打印语句移到 __main__ 中，您就不会再看到它们了.

Just move those print statement within the __main__ and you won't see them again.

这篇关于Python Multiprocessing:并行执行前后串行执行代码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！