Python子进程批量调用 [英] Python Subprocess Batch Call

问题描述

我正在尝试创建一个可变批量大小，以进行子流程调用.我对启动一批 5 个的最佳方法有点困惑，等待所有 5 个完成而不是接下来的五个启动.

I am trying to create a variable batch size, to make a subprocess call. I am a bit confused on the best way to have a batch of 5 kick off, wait for all 5 to complete than kick off the next five.

到目前为止我所拥有的是:

What I have so far is:

batchSize = 5
proccessArray = process.split(",")
processLength = len(proccessArray) - 1
counter1 = 0
for i in range(0, processLength, batchSize):
    for x in range(1, batchSize):
        d = {}
        if counter1 < processLength:
            dgclOutput = inputPath + st + "_" + (i + x) + "output"
            d["process{0}".format(x)] = subprocess.Popen(proccessArray(i + x) + ">>" + dgclOutput, shell=True, stdout=subprocess.PIPE)
            counter1 + 1
        else:
            break

BatchSize 是我一次要执行的批次数.Process Array 是它需要调用的命令列表.进程长度是可能的命令数量.计数器是在达到最大值时踢出循环.

BatchSize is my number of batches I which to go at a time. Process Array is a listing of commands it needs to call. Process length is the amount of possible commands. Counter is to kick out of the loop when it reaches the max.

所以我的第一个循环步骤是批量大小，而不是在字典中创建 5 个 subpoccesses 的内部循环开始.

So my first loop steps in the amount of the batch size, than an inner loop that creates 5 subpoccesses in a dictionary to kick off.

此代码不起作用，有人知道如何使其工作或更好的解决方案吗?

This code does not work, anyone have an idea how to make it work or a better solution?

推荐答案

我认为您可能正在寻找以下方面的内容:

I think you're probably looking for something along these lines:

batchSize = 5
processArray = process.split(",")
for i in xrange(0, len(processArray), batchSize):
    batch = processArray[i:i+batchSize]  # len(batch) <= batchSize
    ps = []
    for process in batch:
        output = "..."
        p = subprocess.Popen(process + ">>" + output, shell=True, stdout=subprocess.PIPE)
        ps.append(p)
    for p in ps:
        p.wait()

这篇关于Python子进程批量调用的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！