如何限制python中活动线程的数量? [英] How do I limit the number of active threads in python?

问题描述

是python的新手，并且在threading上取得了一些进展-正在做一些音乐文件转换，并希望能够利用我机器上的多个内核(每个内核一个活动的转换线程).

Am new to python and making some headway with threading - am doing some music file conversion and want to be able to utilize the multiple cores on my machine (one active conversion thread per core).

class EncodeThread(threading.Thread):
    # this is hacked together a bit, but should give you an idea
    def run(self):
        decode = subprocess.Popen(["flac","--decode","--stdout",self.src],
                            stdout=subprocess.PIPE)
        encode = subprocess.Popen(["lame","--quiet","-",self.dest],
                                stdin=decode.stdout)
        encode.communicate()

# some other code puts these threads with various src/dest pairs in a list

for proc in threads: # `threads` is my list of `threading.Thread` objects
    proc.start()

一切正常，所有文件都经过编码，太棒了！ ...但是，所有进程都立即生成，但是我只想一次运行两个(每个内核一个).一旦完成，我希望它移至列表中的下一个，直到完成，然后继续执行该程序.

Everything works, all the files get encoded, bravo! ... however, all the processes spawn immediately, yet I only want to run two at a time (one for each core). As soon as one is finished, I want it to move on to the next on the list until it is finished, then continue with the program.

我该怎么做?

(我看过线程池和队列函数，但找不到简单的答案.)

(I've looked at the thread pool and queue functions but I can't find a simple answer.)

也许我应该补充一点，我的每个线程都使用subprocess.Popen运行单独的命令行 decoder (flac)，该管道通过管道传送到stdout，并被馈送到命令行 encoder (lame/mp3).

maybe I should add that each of my threads is using subprocess.Popen to run a separate command line decoder (flac) piped to stdout which is fed into a command line encoder (lame/mp3).

推荐答案

如果要限制并行线程的数量，请使用

If you want to limit the number of parallel threads, use a semaphore:

threadLimiter = threading.BoundedSemaphore(maximumNumberOfThreads)

class EncodeThread(threading.Thread):

    def run(self):
        threadLimiter.acquire()
        try:
            <your code here>
        finally:
            threadLimiter.release()

一次启动所有线程.除了maximumNumberOfThreads之外的所有线程都将在threadLimiter.acquire()中等待，并且只有另一个线程通过threadLimiter.release()时，等待线程才会继续.

Start all threads at once. All but maximumNumberOfThreads will wait in threadLimiter.acquire() and a waiting thread will only continue once another thread goes through threadLimiter.release().

这篇关于如何限制python中活动线程的数量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！