我需要在Cython中使用`nogil`吗? [英] Do I need to use `nogil` in Cython

问题描述

我有一些想尽快运行的Cython代码.为此，我需要释放GIL吗?

I have some Cython code that I'd like to run as quickly as possible. Do I need to release the GIL in order to do this?

假设我的代码与此类似:

Let's suppose my code is similar to this:

import numpy as np

# trivial definition just for illustration!
cdef double some_complicated_function(double x) nogil:
    return x

cdef void func(double[:] input) nogil:
    cdef double[:] array = np.zeros_like(input)
    for i in range(array.shape[0]):
        array[i] = some_complicated_function(input[i])

我从 np.zeros_like 行中获得了大量错误消息，类似于:

I get a whole load of error messages from the np.zeros_like line similar to:

nogilcode.pyx:7:40: Calling gil-requiring function not allowed without gil
nogilcode.pyx:7:29: Accessing Python attribute not allowed without gil
nogilcode.pyx:7:27: Accessing Python global or builtin not allowed without gil
nogilcode.pyx:7:40: Constructing Python tuple not allowed without gil
nogilcode.pyx:7:41: Converting to Python object not allowed without gil

我是否需要找到一种在没有GIL的情况下调用 np.zeros_like 的方法?还是找到其他不需要GIL的数组分配方式?

Do I need to find a way of calling np.zeros_like without the GIL? Or find some other way of allocating an array that doesn't require the GIL?

注意:这是一个自我解答的问题，旨在消除对Cython和GIL的一些常见误解(当然，当然也欢迎您回答！)

Note: this is a self-answered question designed to clear up some common misunderstanding about Cython and the GIL (although you're welcome to answer it too, of course!)

推荐答案

否-您可能不需要释放GIL.

No - you probably don't need to release the GIL.

GIL(全局解释器锁)的基本功能是通过确保一次只能运行一个Python线程来确保Python的内部机制不受竞争条件的影响.但是，仅按住GIL不会降低代码速度.

The basic function of the GIL (global interpreter lock) is to ensure that Python's internal mechanisms are not subject to race conditions, by ensuring that only one Python thread is able to run at once. However, simply holding the GIL does not slow your code down.

您应该释放GIL的两个(相关)场合是:

The two (related) occasions when you should release the GIL are:

1. 使用 Cython的并行机制.例如， prange 循环的内容必须为 nogil .

1. Using Cython's parallelism mechanism. The contents of a prange loop for example are required to be nogil.

如果您希望其他(外部)Python线程能够同时运行.

If you want other (external) Python threads to be able to run at the same time.

a.如果您有一个不需要GIL的大型计算/IO密集块，则它可能是礼貌的".发布它，只是使想要执行多线程的代码用户受益.但是，这主要是有用的，而不是必要的.

a. if you have a large computationally/IO-intensive block that doesn't need the GIL then it may be "polite" to release it, just to benefit users of your code who want to do multi-threading. However, this is mostly useful rather than necessary.

b.(非常非常偶尔)，使用简短的 with nogil:pass 块短暂地释放GIL有时会很有用.这是因为Cython不会自发地释放它(与Python不同)，因此，如果您正在等待另一个Python线程来完成任务，则可以避免死锁.除非您使用Cython编译GUI代码，否则此子点可能不适用于您.

b. (very, very occasionally) it's sometimes useful to briefly release the GIL with a short with nogil: pass block. This is because Cython doesn't release it spontaneously (unlike Python) so if you're waiting on another Python thread to complete a task, this can avoid deadlocks. This sub-point probably doesn't apply to you unless you're compiling GUI code with Cython.

---

可以在没有GIL的情况下运行的Cython代码(无需调用Python，仅用于C级数字操作)通常是有效的代码.有时，这给人的印象是相反的事实是正确的，而技巧就是释放GIL，而不是他们正在运行的实际代码.别被这误导了-在有或没有GIL的情况下，您的(单线程)代码都将以相同的速度运行.

---

The sort of Cython code that can run without the GIL (no calls to Python, purely C-level numeric operations) is often the sort of code that runs efficiently. This sometimes gives people the impression that the inverse is true and the trick is releasing the GIL, rather than the actual code they're running. Don't be misled by this - your (single-threaded) code will run the same speed with or without the GIL.

因此，如果您有一个不错的快速Numpy函数，该函数可以对大量数据快速执行所需的操作，但只能通过GIL进行调用，则只需调用它-不会造成任何伤害！

Therefore, if you have a nice fast Numpy function that does exactly what you want quickly on a big chunk of data, but can only be called with the GIL, then just call it - no harm is done!

最后一点:即使在 nogil 块中(例如，在 prange 循环中)，也可以随时根据需要恢复GIL:

As a final point: even within a nogil block (for example a prange loop) you can always get the GIL back if you need it:

with gil:
    ... # small block of GIL requiring code goes here

尽量不要经常执行此操作(获取/释放它需要时间，并且当然只有一个线程可以一次运行此块)，但是同样，这是在需要时执行小型Python操作的好方法.

Try not to do this too often (getting/releasing it takes time, and of course only one thread can be running this block at once) but equally it's a good way of doing small Python operations where needed.

这篇关于我需要在Cython中使用`nogil`吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！