如何限制线程数 [英] How to limit the number of Threads

问题描述

THINGS 变量中存储了 101 个事物.该代码声明了 101 个线程并立即同时执行所有线程.

There are 101 things stored in THINGS variable. The code declares 101 threads and executes them all at once instantly all at the same time.

我想知道我们是否可以将活动线程的数量限制为 12 个.

I wonder if we can limit the number of the active threads to just 12.

起初只有 12 个线程应该选择他们的 12 件事来处理.其余的线程应该等待前 12 个线程完成它们的工作.当前 12 个线程都完成后，接下来的 12 个线程将选择接下来的 12 个要处理的事情.还有一个.

At first only 12 threads should pick their 12 THINGS to process. The rest of the threads should be waiting for the first 12 to finish their jobs. When the first 12 threads are all done then the next 12 threads would pick up the next 12 THINGS to process. And so one.

有可能吗?

import Queue
import threading, time

class MyThread(threading.Thread):
    def __init__(self, theQueue=None):
        threading.Thread.__init__(self)        
        self.theQueue=theQueue

    def run(self):
        thing=self.theQueue.get()
        self.process(thing) 
        self.theQueue.task_done()

    def process(self, thing):
        time.sleep(1)
        print 'processing %s'%thing.name

queue=Queue.Queue()
THINGS = ['Thing%02d'%i for i in range(101)]

THREADS=[]
for thing in THINGS:
    thread=MyThread(theQueue=queue)
    thread.name = thing
    THREADS.append(thread)
    thread.start() 

for thread in THREADS:       
    queue.put(thread)

推荐答案

工作解决方案发布在下面.基本思想是我们只声明与可用 CPU 一样多的线程实例.然后我们继续将任务"(或此处的事物")添加到队列中.一旦任务被添加到队列中，它就会立即被我们在上一步中声明的 Thread 实例之一接收.

The working solution is posted below. The basic idea is that we declare only as many Threads instances as there are available CPUs. Then we proceed by adding the "tasks" (or "things" here) to the Queue. As soon as the task is added to the queue it is being immediately picked up by one of the Thread instances we declared in the previous step.

重要提示:为了使此机制起作用，MyThread.run() 方法应该在 while 循环内运行.否则 MyThread 实例将在完成第一个任务后立即终止.while 循环将在队列中没有任务后自行退出.这就是故事的结局.

Important: In order for this mechanism to work the MyThread.run() method should be running inside of the while loop. Otherwise MyThread instance will be terminated as soon as it completes the very first task. The while loop will exit itself after no tasks in the Queue are left. That is the end of story.

import Queue
import threading, time

class MyThread(threading.Thread):
    def __init__(self, theQueue=None):
        threading.Thread.__init__(self)        
        self.theQueue=theQueue

    def run(self):
        while True:
            thing=self.theQueue.get()
            self.process(thing) 
            self.theQueue.task_done()

    def process(self, thing):
        time.sleep(1)
        print 'processing %s'%thing

queue=Queue.Queue()
THINGS = ['Thing%02d'%i for i in range(101)]
AVAILABLE_CPUS=3

for OneOf in range(AVAILABLE_CPUS):
    thread=MyThread(theQueue=queue)
    thread.start() # thread started. But since there are no tasks in Queue yet it is just waiting.

for thing in THINGS:       
    queue.put(thing) # as soon as task in added here one of available Threads picks it up

这篇关于如何限制线程数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！