使用Jug，如何在多处理模式下运行嵌套循环? [英] With Jug, how to run nested loops in multiprocessing mode?

问题描述

例如，以下代码仅在1个进程中运行任务:

for example, the following code only run the task in 1 process:

from jug import TaskGenerator
evaluate = TaskGenerator(evaluate)

for p0 in [1,2,3]:
for p1 in xrange(10):
    for p2 in xrange(10,20):
         for p3 in [True, False]:
             for p4 in xrange(100):
                results.append(evaluate(p0,p1,p2,p3,p4))

我应该在哪里指示进程数? 谢谢！

where should I indicate the number of processes? thanks!

推荐答案

jug 在这里.

要运行此脚本(假设它是一个名为script.py的文件)，您将运行jug execute script.py命令(而不是python script.py).

To run this script, assuming it is a file called script.py, you will run the jug execute script.py command (and not python script.py).

您可以简单地并行运行其中的多个.如果您使用的是类似Unix的外壳，那么以下操作应该可以正常使用两个处理器:

You can simply run multiple of these in parallel. If you are using a Unix-like shell, the following should work just fine in order to use two processors:

jug execute script.py &
jug execute script.py &

不同的进程将使用文件系统相互通信以划分工作(这是默认设置，如果要使用不共享文件系统的计算机集群，也可以使用redis后端) .

The different processes will communicate with each other using the file system to divide up the work (this is the default, you can also use a redis backend if you want to use a cluster of machines that do not share a file system).

您还可以使用以下命令检查运行过程中的进度:

You can also use the following to check on the progress while they run:

jug status script.py

这篇关于使用Jug，如何在多处理模式下运行嵌套循环?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！