在Python中使用dask将tqdm与延迟执行相结合 [英] combining tqdm with delayed execution with dask in python

问题描述

tqdm和dask都是在python中进行迭代的出色软件包. tqdm实现所需的进度条，而dask实现多线程平台，它们都可以减少迭代过程的麻烦.但是-我很难将它们结合在一起.

tqdm and dask are both amazing packages for iterations in python. While tqdm implements the needed progress bar, dask implements the multi-thread platform and they both can make iteration process less frustrating. Yet - I'm having troubles to combine them both together.

例如，以下代码使用tqdm.trange进度条在dask中实现了延迟执行.事实是，由于delayed快速执行，进度条立即结束，而真正的计算时间工作则在compute部分完成.

For example, the following code implements a delayed execution in dask, with tqdm.trange progress bar. The thing is that since the delayed is performed quickly, the progress bar ends immediately, while the real computation-time effort is done during the compute part.

from dask import delayed,compute
from tqdm import trange
from time import sleep

ct = time()
result= []

def fun(x):
    sleep(x)
    return x

for i in trange(10):
    result.append(delayed(fun)(i))

print compute(result)

如何在compute命令中将进度条附加到实际执行中?

How can I attach the progress bar to the actual execution in compute command?

推荐答案

考虑Dask的进度条

from dask.diagnostics import ProgressBar

with ProgressBar():
    compute(result)

构建您自己的诊断程序

您可以使用此插件体系结构在每个任务结束时获取信号. http://dask.pydata.org/en/latest/diagnostics.html

以下是某人确切执行此操作的示例: https://github.com/tqdm/tqdm/issues/278

Here is an example of someone doing exactly this: https://github.com/tqdm/tqdm/issues/278

这篇关于在Python中使用dask将tqdm与延迟执行相结合的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！