多重处理:使用tqdm显示进度条 [英] Multiprocessing : use tqdm to display a progress bar

问题描述

为了使我的代码更"Pythonic"且更快，我使用"multiprocessing"和一个map函数向其发送a)函数和b)迭代范围.

To make my code more "pythonic" and faster, I use "multiprocessing" and a map function to send it a) the function and b) the range of iterations.

植入的解决方案(即直接在tqdm.tqdm(range(0，30))范围内调用tqdm)不适用于多重处理(如下代码所示).

The implanted solution (i.e., call tqdm directly on the range tqdm.tqdm(range(0, 30)) does not work with multiprocessing (as formulated in the code below).

进度条显示为0到100％(当python读取代码时?)，但是它并不表示map函数的实际进度.

The progress bar is displayed from 0 to 100% (when python reads the code?) but it does not indicate the actual progress of the map function.

如何显示进度条以指示地图"功能在哪一步?

from multiprocessing import Pool
import tqdm
import time

def _foo(my_number):
   square = my_number * my_number
   time.sleep(1)
   return square 

if __name__ == '__main__':
   p = Pool(2)
   r = p.map(_foo, tqdm.tqdm(range(0, 30)))
   p.close()
   p.join()

欢迎任何帮助或建议...

Any help or suggestions are welcome...

推荐答案

发现的解决方案:注意！由于进行了多处理，估计时间(每个循环的迭代次数，总时间等)可能不稳定，但是进度条可以正常工作.

Solution Found : Be careful! Due to multiprocessing, estimation time (iteration per loop, total time, etc.) could be unstable, but the progress bar works perfectly.

注意:Pool的上下文管理器仅在Python 3.3版中可用

Note: Context manager for Pool is only available from Python version 3.3

from multiprocessing import Pool
import time
from tqdm import *

def _foo(my_number):
   square = my_number * my_number
   time.sleep(1)
   return square 

if __name__ == '__main__':
    with Pool(processes=2) as p:
        max_ = 30
        with tqdm(total=max_) as pbar:
            for i, _ in enumerate(p.imap_unordered(_foo, range(0, max_))):
                pbar.update()

这篇关于多重处理:使用tqdm显示进度条的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！