python multiprocess.Pool在stdout中按顺序显示结果 [英] python multiprocess.Pool show results in order in stdout

问题描述

在multiprocessing.Pool中，我试图以相同的顺序显示打印件.

In multiprocessing.Pool I am trying to show my prints in the the same order.

from multiprocessing import Pool
import time
def func(arg):
    time.sleep(0.001)
    print(arg, end=" ")

proc_pool = Pool(4)
proc_pool.map(func, range(30))

输出为:0 1 8 9 10 11 14 15 6 7 16 17 4 5 12 13 18 19 2 3或类似值.它不是按0 1 2 3 ...

The output is: 0 1 8 9 10 11 14 15 6 7 16 17 4 5 12 13 18 19 2 3 or similar. It is not in the order 0 1 2 3 ...

我知道imap可以提供更好的排序方式，但是仍然不是我想要的.我可以重写print函数将其保存到变量中，然后一次全部打印-但我希望在它们完成时立即显示它们-而不是全部完成时显示.

I know that imap can give better ordering ... but it's still not exactly what I want. I could override the print function to save it into a variable and print them all at once - but I would prefer showing them as soon as they complete - not when all complete.

推荐答案

鉴于Pool在工作进程(由操作系统调度)之间分配列表的元素，您无法采取任何方法带有map的保证输出顺序.

Given that a Pool distributes the elements of the list amongst the worker processes (which are scheduled by the operating system), there is no way that you can guarantee output order with map.

(使用map)可能最好的办法是将输入更改为元组(sequence_number, data)的列表，让worker函数返回(sequence_number, result)，然后按sequence_number进行排序.

The best you can probably do (with map) is change the input into a list of tuples (sequence_number, data), have the worker function return (sequence_number, result) and than sort by sequence_number.

如果要在项目完成后立即开始处理，请使用imap或imap_unordered.使用imap将保留与可迭代输入相同的顺序.如果对(sequence_number, result)元组和imap使用相同的技巧，则可以将结果保存在列表中，并在没有序列的情况下将它们打印出来.

If you want to start processing items as soon as they are finished, use imap or imap_unordered. Using imap will preserve the same order as the input iterable. If you use the same trick with the (sequence_number, result) tuple with imap, you can save the results in a list and print them once you have a sequence without gaps.

这篇关于python multiprocess.Pool在stdout中按顺序显示结果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！