PyArray_SimpleNewFromData中的奇怪分段错误 [英] Strange Segmentation Fault in PyArray_SimpleNewFromData

问题描述

我的问题在精神上类似于 PyArray_SimpleNewFromData中的分段错误

My question is similar "in spirit" to Segmentation fault in PyArray_SimpleNewFromData

我有一个看起来像这样的C代码:(原始代码实际上测试了malloc()是否返回NULL)

I have a C code that looks like this: (original code actually tests if malloc() returned NULL)

  1 #include <Python.h>
  2 #include <numpy/arrayobject.h>  // (Not sure if right import)
  3 #include <stdlib.h>
  4 #include <stdio.h>
  5 
  6 double *calculate_dW(npy_intp *dim_w) {
  7         int i;
  8         double* data = (double*)malloc(sizeof(double) * dim_w[0]);
  9         
 10         /* Inserts some dummy data */
 11         for (i = 0; i < dim_w[0]; i++)
 12                 data[i] = i;
 13         
 14         return data;
 15 }

然后是将其包装在函数中的Cython代码:

And then a Cython code that wraps it inside a function:

  1 import cython
  2 import numpy as np
  3 cimport numpy as np
  4 
  5 cdef extern double *calculate_dW(np.npy_intp *dim_w)
  6 
  7 def run_calculate_dW(np.ndarray[np.npy_intp, ndim=1, mode="c"] dim_w):
  8         print("Will call calculate_dW")
  9         cdef double *dW = calculate_dW(&dim_w[0])
 10 
 11         print("Will call PyArray_SimpleNewFromData")
 12         ret = np.PyArray_SimpleNewFromData(
 13                 1,
 14                 &dim_w[0],
 15                 np.NPY_FLOAT64,
 16                 dW)
 17         print("Will print")
 18         print(ret)
 19         print("Will return")
 20         return ret

我与哪个进行测试

  # runTest.py
  1 import numpy as np
  2 import multiply
  3 a = np.array((10,)) # as expected, using `np.array(10)` won't work
  4 print a
  5 multiply.run_calculate_dW(a)

并获得以下输出

$ PYTHONPATH=build/lib.linux-x86_64-2.7/ python runTest.py 
[10]
Will call calculate_dW
Will call PyArray_SimpleNewFromData
Segmentation fault (core dumped)

(即，调用PyArray_SimpleNewFromData()时出现了SegFault(如果我将其替换为ret = 1，则细分错误将消失).调试时，我尝试了很多事情:

(i.e., a SegFault in the call to PyArray_SimpleNewFromData() (if I replace it by, say, ret = 1, the Segmentation Fault vanishes). When debugging, I tried many things:

• 将尺寸数更改为1；
• 增加malloc()分配的内存量(以确保我没有访问任何我不应该访问的内容)；
• 将np.NPY_FLOAT32更改为np.float32;
• 改变我传递新数组形状"的方式.

• Changing the number of dimensions to 1;
• Increasing the amount of memory allocated by malloc() (to guarantee I was not accessing anything I shouldn't);
• Changing np.NPY_FLOAT32 to np.float32;
• Changing the way I pass the "shape" of the new array.

我相信我正在严格按照文档，以及其他问题的答案.我似乎没有收到任何编译器错误或警告.

I believe I am following precisely the documentation, as well as the answer to this other question. I don't seem to get any compiler error or warning.

不过，我确实注意到互联网上的所有其他代码在调用PyArray_SimpleNewFromData时都使用C(而不是Python).我尝试从C函数返回PyObject*，但无法使其编译.

Still, I do have noticed that all other codes around in the internet are using C (instead of Python) when they call PyArray_SimpleNewFromData. I tried returning a PyObject* from the C function, but couldn't get it to compile.

此外，我确实收到一些使用不赞成使用的NumPy API，请通过#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION禁用它"警告；但我读过，我可以放心地忽略它们. ( Cython Numpy关于NPY_NO_DEPRECATED_API的警告)

Also, I do get some "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" warning; but I have read I am safe to ignore them. (Cython Numpy warning about NPY_NO_DEPRECATED_API when using MemoryView )

有什么建议吗? (还有，从dW中创建numpy数组的任何其他方法吗?)

Any suggestion? (also, any other way of creating a numpy array out of dW?)

推荐答案

我认为问题是，当您希望将指针指向整数时，您正在将Python列表作为PyArray_SimpleNewFromData的第二个参数传递.我对此编译感到有些惊讶.

I think the issue is that you're passing a Python list as the second argument to PyArray_SimpleNewFromData when it expects a pointer to an integer. I'm a little surprised this compiles.

尝试:

ret = np.PyArray_SimpleNewFromData(
                     4,
                     &dim_w[0], # pointer to first element
                     np.NPY_FLOAT64,
                     dW)

请注意，我也将类型更改为NPY_FLOAT64，因为它应该与double相匹配.

Note that I've also changed the type to NPY_FLOAT64 since that should match double.

我还将dim_w的定义更改为

np.ndarray[np.NPY_INTP, ndim=1, mode="c"] dim_w

以确保数组的类型与numpy期望的匹配.这可能还需要将calculate_dW的签名更改为double *calculate_dW(intptr_t *dim_w)也要匹配.

to ensure that the type of the array matches what numpy is expecting. This may also require changing the signature of calculate_dW to double *calculate_dW(intptr_t *dim_w) to match too.

编辑:第二个问题是您需要添加该行

A second issue is that you need to include the line

np.import_array()

您的Cython文件中的

(在导入后位于顶层).这会为numpy做一些设置.原则上，我认为文档建议您在执行cimport numpy时始终将其包括在内.在实践中，这只是有时很重要，而这只是其中一次.

in your Cython file (just at the top level, after your imports). This does some setup stuff for numpy. In principle I think the documentation recommends you always include it when doing cimport numpy. In practice it only sometimes matter, and this is one of those times.

(现在已经测试了答案)

(Answer is now tested)

这篇关于PyArray_SimpleNewFromData中的奇怪分段错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！