没有GIL的cython memoryviews切片 [英] cython memoryviews slices without GIL

问题描述

我想释放GIL，以并行化cython中的循环，在该循环中，不同的memoryviews片段将传递给循环内的某个函数.代码如下:

I want to release the GIL in order to parallelise loop in cython, where different slices of memoryviews are passed to a some function inside the loop. The code looks like this:

cpdef void do_sth_in_parallel(bint[:,:] input, bint[:] output, int D):
     for d in prange(D, schedule=dynamic, nogil=True):
          ouput[d] = some_function_not_requiring_gil(x[d,:])

这是不可能的，因为选择切片x [d ,:]似乎需要GIL.运行 cython -a ，并使用普通的for循环，我得到下面的代码.如何在纯C语言中完成此操作?

This is not possible, since selecting the slice x[d,:], seems to require GIL. Running cython -a, and using a normal for loop, I get the code posted below. How can this be done in pure C?

      __pyx_t_5.data = __pyx_v_x.data;
      __pyx_t_5.memview = __pyx_v_x.memview;
      __PYX_INC_MEMVIEW(&__pyx_t_5, 0);
      {
    Py_ssize_t __pyx_tmp_idx = __pyx_v_d;
    Py_ssize_t __pyx_tmp_shape = __pyx_v_x.shape[0];
    Py_ssize_t __pyx_tmp_stride = __pyx_v_x.strides[0];
    if (0 && (__pyx_tmp_idx < 0))
        __pyx_tmp_idx += __pyx_tmp_shape;
    if (0 && (__pyx_tmp_idx < 0 || __pyx_tmp_idx >= __pyx_tmp_shape)) {
        PyErr_SetString(PyExc_IndexError, "Index out of bounds (axis 0)");
        __PYX_ERR(0, 130, __pyx_L1_error)
    }
        __pyx_t_5.data += __pyx_tmp_idx * __pyx_tmp_stride;
}

__pyx_t_5.shape[0] = __pyx_v_x.shape[1];
__pyx_t_5.strides[0] = __pyx_v_x.strides[1];
    __pyx_t_5.suboffsets[0] = -1;

__pyx_t_6.data = __pyx_v_u.data;
      __pyx_t_6.memview = __pyx_v_u.memview;
      __PYX_INC_MEMVIEW(&__pyx_t_6, 0);
      __pyx_t_6.shape[0] = __pyx_v_u.shape[0];
__pyx_t_6.strides[0] = __pyx_v_u.strides[0];
    __pyx_t_6.suboffsets[0] = -1;

推荐答案

以下对我有用:

from cython.parallel import prange

cdef bint some_function_not_requiring_gil(bint[:] x) nogil:
    return x[0]

cpdef void do_sth_in_parallel(bint[:,:] input, bint[:] output, int D):
     cdef int d
     for d in prange(D, schedule=dynamic, nogil=True):
          output[d] = some_function_not_requiring_gil(input[d,:])

我必须做的两个主要更改是将x更改为input(因为假设它可以在全局范围内找到x作为python对象)

The two main changes I had to make were x to input (because it's assuming it can find x as a python object at the global scope) to fix the error

在没有gil的情况下不允许转换为Python对象

Converting to Python object not allowed without gil

并添加cdef int d以强制d的类型并修复错误

and adding cdef int d to force the type of d and fix the error

在没有GIL的情况下不允许从Python强制转换

Coercion from Python not allowed without the GIL

(我还创建了一个示例some_function_not_requiring_gil，但我认为这很明显)

(I also created an example some_function_not_requiring_gil but I assume this is fairly obvious)

这篇关于没有GIL的cython memoryviews切片的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！