Numpy C API:链接多个目标文件 [英] Numpy C API: Link several object files
问题描述
我正在使用 numpy 的 C API 编写一些用于矩阵计算的函数.今天我想将我的函数的某些部分移动到一个单独的 .c 文件中，并使用标头来声明它们.现在我有一个奇怪的问题，它与 numpy 的 import_array 函数有关.我试图尽可能地简化问题.首先是工作程序:
I'm using the C API of numpy to write some functions for matrix calculation. Today I wanted to move some parts of my functions into a seperate .c file and use a header to declare them. Now I have a strange problem that has to do with numpy's import_array function. I've tried to simplify the problem as much as possible. At first there is the working program:
mytest.c
#include "mytest.h"
PyObject* my_sub_function() {
npy_intp dims[2] = {2, 2};
double data[] = {0.1, 0.2, 0.3, 0.4};
PyArrayObject* matrix = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_FLOAT64);
memcpy(PyArray_DATA(matrix), data, sizeof(double) * dims[0] * dims[1]);
return (PyObject*)matrix;
}
static PyObject* my_test_function(PyObject* self, PyObject* args) {
return my_sub_function();
}
static PyMethodDef methods[] = {
{"my_test_function", my_test_function, METH_VARARGS, ""},
{0, 0, 0, 0}
};
static struct PyModuleDef module = {
PyModuleDef_HEAD_INIT, "mytest", 0, -1, methods
};
PyMODINIT_FUNC PyInit_mytest() {
import_array();
return PyModule_Create(&module);
}
mytest.h
#ifndef mytest_h
#define mytest_h
#include <Python.h>
#include <numpy/arrayobject.h>
#include <numpy/npy_common.h>
PyObject* my_sub_function();
#endif
Makefile
all: mytest.o sub.o
gcc -shared -Wl,-soname,mytest.so -o mytest.so mytest.o
mytest.o: sub.o
gcc -fPIC -c mytest.c `pkg-config --cflags python3`
clean:
rm -rf *.so
rm -rf *.o
一切都按预期进行.我可以调用 make 然后加载模块并调用函数:
Everything works as expected. I can call make and then load the module and call the function:
test.py
import mytest
print(mytest.my_test_function())
如果我从 init 函数中删除 import_array 就会出现段错误，这是许多邮件列表和论坛中报告的行为.
If I would remove import_array from the init function there would be a segfault, which is the behaviour that has been reported in many mailing lists and forums.
现在我只想从 mytest.c 中删除整个函数 my_sub_function 并将其移动到名为 sub.c 的文件中:
Now I just want to remove the whole function my_sub_function from mytest.c and move it into a file called sub.c:
#include "mytest.h"
PyObject* my_sub_function() {
npy_intp dims[2] = {2, 2};
double data[] = {0.1, 0.2, 0.3, 0.4};
PyArrayObject* matrix = (PyArrayObject*)PyArray_SimpleNew(2, dims, NPY_FLOAT64);
memcpy(PyArray_DATA(matrix), data, sizeof(double) * dims[0] * dims[1]);
return (PyObject*)matrix;
}
新的Makefile是:
all: mytest.o sub.o
gcc -shared -Wl,-soname,mytest.so -o mytest.so mytest.o sub.o
mytest.o:
gcc -fPIC -c mytest.c `pkg-config --cflags python3`
sub.o:
gcc -fPIC -c sub.c `pkg-config --cflags python3`
clean:
rm -rf *.so
rm -rf *.o
如果我现在尝试加载模块并调用函数，函数调用会给我一个段错误.如果我把对 import_array 的调用放在 my_sub_function 的顶部，我可以解决这个问题，但我认为这不是函数应该使用的方式.
If I try to load the module and to call the function now the function call gives me a segfault. I can resolve the problem if I put a call to import_array to the top of my_sub_function, but I don't think that this is the way that function should be used.
所以我想知道为什么会发生这种情况，以及将 numpy 模块拆分为多个源文件的干净"方式是什么.
So I'd like to know why this is happening and what's the "clean" way to split up a numpy module into several source files.
推荐答案
默认情况下，import_array 例程只会使 NumPy C API 在单个文件中可用.这是因为它通过存储在静态全局变量中的函数指针表工作(即不导出，仅在同一文件中可见).
By default, the import_array routine will only make the NumPy C API available within a single file. This is because it works through a table of function pointers stored in a static global variable (i.e. not exported, and only visible within the same file).
正如文档中提到的，您可以通过一些预处理器定义来改变这种行为:
As mentioned in the documentation, you can change this behaviour with a few preprocessor definitions:
在您的扩展程序的所有文件中，将
PY_ARRAY_UNIQUE_SYMBOL定义为一个不太可能与其他扩展程序冲突的唯一变量.在变量名称中包含扩展程序的模块名称是个好主意.
In all files for your extension, define
PY_ARRAY_UNIQUE_SYMBOLto a unique variable that is unlikely to conflict with other extensions. Including your extension's module name in the variable name would be a good idea.
在除了调用 import_array 的文件之外的每个文件中，定义符号 NO_IMPORT_ARRAY
In every file except for the one where you call import_array, define the symbol NO_IMPORT_ARRAY
这些符号需要在包含 arrayobject.h 之前定义，以便它们生效.
These symbols need to be defined before you include arrayobject.h in order for them to take effect.
这篇关于Numpy C API:链接多个目标文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
