python 队列，SimpleQueue，task_done [英] python Queue, SimpleQueue, task_done

问题描述

1. 当我使用 queue.Queue() 时，我必须调用 task_done() 吗?

2. 如果我不需要任务计数，是否应该使用 queue.SimpleQueue 代替(以获得更好的性能)?我不确定 SimpleQueue 是否是线程安全的?因为我看到 put() 实现没有获取锁?

3. 当我在 Queue() 实现中播种时，如果我不调用 task_done()，则会增加一个计数器，并且由于我的程序长时间运行，不调用task_done() 会导致内存泄漏吗?因为数太多.

解决方案

1. 除非您使用 Queue.join() 函数，否则您不必调用 task_done().

<块引用>

Queue.join() 阻塞直到队列中的所有项目都被获取和处理.

<块引用>

每当将项目添加到队列时，未完成任务的计数就会增加.每当消费者线程调用 task_done() 以指示该项目已被检索并且其上的所有工作已完成时，计数就会下降.当未完成任务的数量降为零时，join() 解除阻塞.

2. 是的，如果您不需要跟踪功能，您可以使用 queue.SimpleQueue 而不是 queue.Queue 作为更轻量级的版本(task_done, join).SimpleQueue 是线程安全的，而且更多，正如 在此处回答

<块引用>

它处理重入 - 在可能会中断同一线程中的其他工作的不稳定情况下调用 queue.SimpleQueue.put 是安全的.例如，您可以安全地从 __del__ 方法、weakref 回调或信号模块信号处理程序调用它.

注意:至少这适用于它的 С 实现.

3. 据我所知，queue.Queue 使用 int 作为 put 的计数器.在我看来，这个计数器会占用太多内存的可能性更多的是理论而不是实际.

1. Do I have to call task_done() when I use queue.Queue()?

2. If I don't need task counting, should I use queue.SimpleQueue instead (for a better performance)? I am not sure if SimpleQueue is thread-safe? cause I see that the put() implementation does not acquire a lock?

3. As I seed in Queue() implementation, if I won't call task_done(), there is a counter which is incremented, and since my program is long running, not calling task_done() will cause some memory leak? because of counting up too high number.

解决方案

1. You don't have to call task_done() unless you use Queue.join() function.

Queue.join() blocks until all items in the queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate that the item was retrieved and all work on it is complete. When the count of unfinished tasks drops to zero, join() unblocks.

2. Yes, you can use queue.SimpleQueue instead of queue.Queue as more light-weight version if you don't need the functionality of tracking (task_done, join). SimpleQueue is thread-safe and more, as answered here

It handles reentrancy - it is safe to call queue.SimpleQueue.put in precarious situations where it might be interrupting other work in the same thread. For example, you can safely call it from __del__ methods, weakref callbacks, or signal module signal handlers.

Note: At least this applies to its С implementation.

3. As I know, queue.Queue uses int as a counter for put. And the possibility that this counter will occupy too much memory is of more theoretical rather than practical, in my humble opinion.

这篇关于python 队列，SimpleQueue，task_done的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！