Python中的队列与JoinableQueue [英] Queue vs JoinableQueue in Python

问题描述

在 Python 中使用多处理模块时有两种队列:

• 队列
• JoinableQueue.

它们之间有什么区别?

队列

来自多处理导入队列q = 队列()q.put(item) # 将一个项目放入队列item = q.get() # 从队列中获取一项

JoinableQueue

from multiprocessing import JoinableQueueq = JoinableQueue()q.task_done() # 信号任务完成q.join() # 等待完成

解决方案

JoinableQueue 有方法 join() 和 task_done()，它们Queue 没有.

<小时><块引用>

class multiprocessing.Queue( [maxsize] )

返回一个使用管道和一些锁/信号量实现的进程共享队列.当一个进程第一次将一个项目放入队列时，一个馈线线程就会启动，它将对象从缓冲区传输到管道中.

标准库的 Queue 模块中常见的 Queue.Empty 和 Queue.Full 异常会引发超时.

Queue 实现了 Queue.Queue 的所有方法，除了 task_done() 和 join().

<小时><块引用>

class multiprocessing.JoinableQueue( [maxsize] )

JoinableQueue 是 Queue 的一个子类，是一个额外具有 task_done() 和 join() 方法的队列.

task_done()

表示以前排队的任务已完成.由队列消费者线程使用.对于用于获取任务的每个 get()，随后对 task_done() 的调用告诉队列该任务的处理已完成.

如果 join() 当前处于阻塞状态，它将在所有项目都处理完毕后恢复(这意味着对于已 put() 进入队列的每个项目都收到了 task_done() 调用).

如果调用次数超过队列中放置的项目数，则会引发 ValueError.

join()

阻塞直到队列中的所有项目都被获取和处理.

每当将项目添加到队列时，未完成任务的计数就会增加.每当消费者线程调用 task_done() 以指示该项目已被检索并且其上的所有工作已完成时，计数就会下降.当未完成任务的数量降为零时，join() 解除阻塞.

<小时>

如果你使用 JoinableQueue 那么你必须为每个从队列中移除的任务调用 JoinableQueue.task_done() 否则用于计算未完成任务数量的信号量可能会最终溢出，引发异常.

In Python while using multiprocessing module there are 2 kinds of queues:

• Queue
• JoinableQueue.

What is the difference between them?

Queue

from multiprocessing import Queue
q = Queue()
q.put(item) # Put an item on the queue
item = q.get() # Get an item from the queue

JoinableQueue

from multiprocessing import JoinableQueue
q = JoinableQueue()
q.task_done() # Signal task completion
q.join() # Wait for completion

解决方案

JoinableQueue has methods join() and task_done(), which Queue hasn't.

---

class multiprocessing.Queue( [maxsize] )

Returns a process shared queue implemented using a pipe and a few locks/semaphores. When a process first puts an item on the queue a feeder thread is started which transfers objects from a buffer into the pipe.

The usual Queue.Empty and Queue.Full exceptions from the standard library’s Queue module are raised to signal timeouts.

Queue implements all the methods of Queue.Queue except for task_done() and join().

---

class multiprocessing.JoinableQueue( [maxsize] )

JoinableQueue, a Queue subclass, is a queue which additionally has task_done() and join() methods.

task_done()

Indicate that a formerly enqueued task is complete. Used by queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Raises a ValueError if called more times than there were items placed in the queue.

join()

Block until all items in the queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.

---

If you use JoinableQueue then you must call JoinableQueue.task_done() for each task removed from the queue or else the semaphore used to count the number of unfinished tasks may eventually overflow, raising an exception.

这篇关于Python中的队列与JoinableQueue的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！