为什么 Python threading.Condition() notify() 需要锁? [英] Why does Python threading.Condition() notify() require a lock?

问题描述

我的问题特别提到了为什么它是这样设计的，因为不必要的性能影响.

My question refers specifically to why it was designed that way, due to the unnecessary performance implication.

当线程 T1 有这个代码时:

When thread T1 has this code:

cv.acquire()
cv.wait()
cv.release()

和线程 T2 有这个代码:

and thread T2 has this code:

cv.acquire()
cv.notify()  # requires that lock be held
cv.release()

发生的事情是 T1 等待并释放锁，然后 T2 获取它，通知 cv 唤醒 T1.现在，从 wait() 返回后，T2 的释放和 T1 的重新获取之间存在竞争条件.如果 T1 先尝试重新获取，它将被不必要地重新挂起，直到 T2 的 release() 完成.

what happens is that T1 waits and releases the lock, then T2 acquires it, notifies cv which wakes up T1. Now, there is a race-condition between T2's release and T1's reacquiring after returning from wait(). If T1 tries to reacquire first, it will be unnecessarily resuspended until T2's release() is completed.

注意:我故意不使用 with 语句，以便更好地说明显式调用的竞争.

Note: I'm intentionally not using the with statement, to better illustrate the race with explicit calls.

这似乎是一个设计缺陷.是否有任何已知的理由，或者我遗漏了什么?

This seems like a design flaw. Is there any rationale known for this, or am I missing something?

推荐答案

这不是一个确定的答案，但它应该涵盖我设法收集的有关此问题的相关详细信息.

This is not a definitive answer, but it's supposed to cover the relevant details I've managed to gather about this problem.

首先，Python 的线程实现是基于 Java 的.Java 的 Condition.signal() 文档内容如下:

First, Python's threading implementation is based on Java's. Java's Condition.signal() documentation reads:

当调用此方法时，实现可能(并且通常确实会)要求当前线程持有与此 Condition 关联的锁.

An implementation may (and typically does) require that the current thread hold the lock associated with this Condition when this method is called.

现在，问题是为什么要特别在 Python 中强制这种行为.但首先我想介绍每种方法的优缺点.

Now, the question was why enforce this behavior in Python in particular. But first I want to cover the pros and cons of each approach.

至于为什么有些人认为持有锁通常更好，我发现了两个主要论点:

As to why some think it's often a better idea to hold the lock, I found two main arguments:

1. 从服务员 acquire() 拿到锁的那一刻开始——也就是说，在 wait() 上释放它之前——它保证被通知信号.如果相应的 release() 在发信号之前发生，这将允许序列(其中 P=Producer 和 C=Consumer)P:释放()；C:获取()；P:通知()；C: wait() 在这种情况下，对应于同一流的 <​​code>acquire() 的 wait() 将错过信号.有些情况下这无关紧要(甚至可以被认为更准确)，但有些情况下这是不可取的.这是一个论点.

1. From the minute a waiter acquire()s the lock—that is, before releasing it on wait()—it is guaranteed to be notified of signals. If the corresponding release() happened prior to signalling, this would allow the sequence(where P=Producer and C=Consumer) P: release(); C: acquire(); P: notify(); C: wait() in which case the wait() corresponding to the acquire() of the same flow would miss the signal. There are cases where this doesn't matter (and could even be considered to be more accurate), but there are cases where that's undesirable. This is one argument.

当你在锁外notify()，这可能会导致调度优先级倒置；也就是说，低优先级线程最终可能会优先于高优先级线程.考虑一个有一个生产者和两个消费者(LC=低优先级消费者和HC=高优先级消费者)的工作队列，其中LC是当前正在执行一个工作项并且 HC 在 wait() 中被阻止.

When you notify() outside a lock, this may cause a scheduling priority inversion; that is, a low-priority thread might end up taking priority over a high-priority thread. Consider a work queue with one producer and two consumers (LC=Low-priority consumer and HC=High-priority consumer), where LC is currently executing a work item and HC is blocked in wait().

可能会出现以下顺序:

P                    LC                    HC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                     execute(item)                   (in wait())
lock()                                  
wq.push(item)
release()
                     acquire()
                     item = wq.pop()
                     release();
notify()
                                                     (wake-up)
                                                     while (wq.empty())
                                                       wait();

如果 notify() 发生在 release() 之前，LC 将无法 acquire() 在 HC 被唤醒之前.这就是发生优先级反转的地方.这是第二个参数.

Whereas if the notify() happened before release(), LC wouldn't have been able to acquire() before HC had been woken-up. This is where the priority inversion occurred. This is the second argument.

支持在锁外进行通知的论点是为了高性能线程，在这种情况下，线程不需要为了在它获得的下一个时间片再次唤醒而返回休眠状态——这已经解释了它是如何实现的可能会发生在我的问题中.

The argument in favor of notifying outside of the lock is for high-performance threading, where a thread need not go back to sleep just to wake-up again the very next time-slice it gets—which was already explained how it might happen in my question.

在 Python 中，正如我所说，您必须在通知时保持锁定.具有讽刺意味的是，内部实现不允许底层操作系统避免优先级反转，因为它对等待程序强制执行 FIFO 顺序.当然，服务员的顺序是确定性的这一事实可能会派上用场，但问题仍然是为什么要强制执行这样的事情，因为有人会争辩说区分锁和条件变量会更精确，因为在一些需要优化并发和最小阻塞的流，acquire() 不应该自己注册一个前面的等待状态，而应该只注册 wait() 调用本身.

In Python, as I said, you must hold the lock while notifying. The irony is that the internal implementation does not allow the underlying OS to avoid priority inversion, because it enforces a FIFO order on the waiters. Of course, the fact that the order of waiters is deterministic could come in handy, but the question remains why enforce such a thing when it could be argued that it would be more precise to differentiate between the lock and the condition variable, for that in some flows that require optimized concurrency and minimal blocking, acquire() should not by itself register a preceding waiting state, but only the wait() call itself.

可以说，Python 程序员无论如何都不会关心性能到这种程度——尽管这仍然没有回答为什么在实现标准库时不应该允许多个标准行为成为可能的问题.

Arguably, Python programmers would not care about performance to this extent anyway—although that still doesn't answer the question of why, when implementing a standard library, one should not allow several standard behaviors to be possible.

还有一点要说的是，threading 模块的开发人员可能出于某种原因特别想要一个 FIFO 顺序，并发现这是实现它的最佳方式，并且想要将其建立为 Condition 以牺牲其他(可能更流行的)方法为代价.为此，在他们自己解释之前，他们应该得到怀疑的好处.

One thing which remains to be said is that the developers of the threading module might have specifically wanted a FIFO order for some reason, and found that this was somehow the best way of achieving it, and wanted to establish that as a Condition at the expense of the other (probably more prevalent) approaches. For this, they deserve the benefit of the doubt until they might account for it themselves.

这篇关于为什么 Python threading.Condition() notify() 需要锁?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！