Numpy C API：链接几个对象文件 [英] Numpy C API: Link several object files

问题描述

我使用numpy的C API编写矩阵计算的一些函数。今天，我想将我的函数的一些部分移动到一个单独的.c文件中，并使用头部来声明它们。现在我有一个奇怪的问题，与numpy的 import_array 函数有关。我试图尽可能简化问题。首先是工作程序：

mytest.c

  #includemytest.h
 
 PyObject * my_sub_function（）{
 npy_intp dims [2] = {2，2}; 
 double data [] = {0.1,0.2,0.3,0.4}; 
 
 PyArrayObject * matrix =（PyArrayObject *）PyArray_SimpleNew（2，dims，NPY_FLOAT64）; 
 memcpy（PyArray_DATA（matrix），data，sizeof（double）* dims [0] * dims [1]）; 
 
 return（PyObject *）matrix; 
 
 
 static PyObject * my_test_function（PyObject * self，PyObject * args）{
 return my_sub_function（）; 
 
 
静态PyMethodDef方法[] = {
 {my_test_function，my_test_function，METH_VARARGS，}，
 {0,0,0,0} 
}; 
 
静态结构PyModuleDef模块= {
 PyModuleDef_HEAD_INIT，mytest，0，-1，methods 
}; 
 
 PyMODINIT_FUNC PyInit_mytest（）{
 import_array（）; 
返回PyModule_Create（& module）; 
} 
  

mytest.h

  #ifndef mytest_h 
 #define mytest_h 
 
 #include< Python.h> 
 #include< numpy / arrayobject.h> 
 #include< numpy / npy_common.h> 
 
 PyObject * my_sub_function（）; 
 
 #endif 
  

Makefile

  all：mytest.o sub.o 
 gcc -shared -Wl，-soname，mytest.so -o mytest.so mytest.o 
 
 mytest.o：sub.o 
 gcc -fPIC -c mytest.c`pkg-config --cflags python3` 
 
 clean：
 rm -rf * .so 
 rm -rf * .o 
  

一切正常如预期。我可以调用 make ，然后加载模块并调用函数：

test.py

$ p $ import mytest
print（mytest.my_test_function（））


如果我从init函数中删除 import_array ，那么会出现段错误，行为已在许多邮件列表和论坛中报告。

现在我只想删除整个函数 my_sub_function 从 mytest.c 移动到名为 sub.c 的文件中：

  #includemytest.h
 
 PyObject * my_sub_function（）{
 npy_intp dims [2] = {2，2}; 
 double data [] = {0.1,0.2,0.3,0.4}; 
 
 PyArrayObject * matrix =（PyArrayObject *）PyArray_SimpleNew（2，dims，NPY_FLOAT64）; 
 memcpy（PyArray_DATA（matrix），data，sizeof（double）* dims [0] * dims [1]）; 
 
 return（PyObject *）matrix; 
 
  

新的 Makefile 为：

  all：mytest.o sub.o 
 gcc -shared -Wl，-soname，mytest.so -o mytest.so mytest .o sub.o 
 
 mytest.o：
 gcc -fPIC -c mytest.c`pkg-config --cflags python3` 
 
 sub.o： 
 gcc -fPIC -c sub.c`pkg-config --cflags python3` 
 
 clean：
 rm -rf * .so 
 rm -rf *。 o 
  

如果我尝试加载模块并调用函数，那么函数调用会给我一个段错误。如果我将 import_array 调用到 my_sub_function 的顶部，我可以解决问题，但我不认为这是应该使用函数的方式。

所以我想知道为什么会发生这种情况，以及干净的方式来分割一个numpy模块分成几个源文件。

解决方案

默认情况下， import_array 只能在单个文件中提供NumPy C API。这是因为它通过存储在静态全局变量中的函数指针表来工作（即，不导出，并且仅在同一文件中可见）。

As 在文档中提到，你可以用一些预处理器定义：

1. 在您的扩展的所有文件中，定义 PY_ARRAY_UNIQUE_SYMBOL 到一个不太可能与其他扩展冲突的独特变量。




2. 除了您调用 import_array的文件之外的每个文件中都包含扩展名的模块名称，定义符号 NO_IMPORT_ARRAY




在包含 arrayobject.h 以使它们生效之前，需要定义这些符号。

I'm using the C API of numpy to write some functions for matrix calculation. Today I wanted to move some parts of my functions into a seperate .c file and use a header to declare them. Now I have a strange problem that has to do with numpy's import_array function. I've tried to simplify the problem as much as possible. At first there is the working program:

mytest.c

#include "mytest.h"

PyObject* my_sub_function() {
    npy_intp dims[2] = {2, 2};
    double data[] = {0.1, 0.2, 0.3, 0.4};

    PyArrayObject* matrix = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_FLOAT64);
    memcpy(PyArray_DATA(matrix), data, sizeof(double) * dims[0] * dims[1]);

    return (PyObject*)matrix;
}

static PyObject* my_test_function(PyObject* self, PyObject* args) {
    return my_sub_function();
}

static PyMethodDef methods[] = {
    {"my_test_function", my_test_function, METH_VARARGS, ""},
    {0, 0, 0, 0}
};

static struct PyModuleDef module = {
    PyModuleDef_HEAD_INIT, "mytest", 0, -1, methods
};

PyMODINIT_FUNC PyInit_mytest() {
    import_array();
    return PyModule_Create(&module);
}

mytest.h

#ifndef mytest_h
#define mytest_h

#include <Python.h>
#include <numpy/arrayobject.h>
#include <numpy/npy_common.h>

PyObject* my_sub_function();

#endif

Makefile

all: mytest.o sub.o
    gcc -shared -Wl,-soname,mytest.so -o mytest.so mytest.o

mytest.o: sub.o
    gcc -fPIC -c mytest.c `pkg-config --cflags python3`

clean:
    rm -rf *.so
    rm -rf *.o

Everything works as expected. I can call make and then load the module and call the function:

test.py

import mytest
print(mytest.my_test_function())

If I would remove import_array from the init function there would be a segfault, which is the behaviour that has been reported in many mailing lists and forums.

Now I just want to remove the whole function my_sub_function from mytest.c and move it into a file called sub.c:

#include "mytest.h"

PyObject* my_sub_function() {
    npy_intp dims[2] = {2, 2};
    double data[] = {0.1, 0.2, 0.3, 0.4};

    PyArrayObject* matrix = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_FLOAT64);
    memcpy(PyArray_DATA(matrix), data, sizeof(double) * dims[0] * dims[1]);

    return (PyObject*)matrix;
}

The new Makefile is:

all: mytest.o sub.o
    gcc -shared -Wl,-soname,mytest.so -o mytest.so mytest.o sub.o

mytest.o:
    gcc -fPIC -c mytest.c `pkg-config --cflags python3`

sub.o:
    gcc -fPIC -c sub.c `pkg-config --cflags python3`

clean:
    rm -rf *.so
    rm -rf *.o

If I try to load the module and to call the function now the function call gives me a segfault. I can resolve the problem if I put a call to import_array to the top of my_sub_function, but I don't think that this is the way that function should be used.

So I'd like to know why this is happening and what's the "clean" way to split up a numpy module into several source files.

解决方案

By default, the import_array routine will only make the NumPy C API available within a single file. This is because it works through a table of function pointers stored in a static global variable (i.e. not exported, and only visible within the same file).

As mentioned in the documentation, you can change this behaviour with a few preprocessor definitions:

1. In all files for your extension, define PY_ARRAY_UNIQUE_SYMBOL to a unique variable that is unlikely to conflict with other extensions. Including your extension's module name in the variable name would be a good idea.

2. In every file except for the one where you call import_array, define the symbol NO_IMPORT_ARRAY

These symbols need to be defined before you include arrayobject.h in order for them to take effect.

这篇关于Numpy C API：链接几个对象文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！