Segfault时import_array不在同一个翻译单元 [英] Segfault when import_array not in same translation unit
问题描述
我无法正确初始化NumPy C API。我想我已经隔离了问题，从不同的翻译单元调用 import_array ，但我不知道为什么这应该是重要。
最小工作示例：
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#include< Python.h>
#include< numpy / npy_3kcompat.h>
#include< numpy / arrayobject.h>
void initialize（）;
#endif
file1.cpp / p>
#includeheader1.hpp
void * wrap_import_array（）
{
import_array（）;
return（void *）1;
}
void initialize（）
{
wrap_import_array（）;
}
file2.cpp
$ b
#includeheader1.hpp
#include< iostream>
void * loc_wrap_import_array（）
{
import_array（）;
return（void *）1;
}
void loc_initialize（）
{
loc_wrap_import_array（）;
}
int main（）
{
Py_Initialize（）;
#ifdef USE_LOC_INIT
loc_initialize（）;
#else
initialize（）;
#endif
npy_intp dms [] = {5};
std :: cout<< creation descr<< std :: endl;
PyArray_Descr * dtype = PyArray_DescrFromType（NPY_FLOAT64）;
std :: cout<< zeros<< std :: endl;
PyArray_Zeros（1，dims，dtype，0）;
std :: cout<< 清理< std :: endl;
return 0;
}
编译器命令：
g ++ file1.cpp file2.cpp -o segissue -lpython3.4m -I / usr / include / python3.4m -DUSE_LOC_INIT
./segissue
＃runs罚款
g ++ file1.cpp file2.cpp -o segissue -lpython3.4m -I / usr / include / python3.4m
./segissue
＃segfaults
我用Clang 3.6.0，GCC 4.9.2，Python 2.7和Python 3.4测试了这个适当修改 wrap_import_array ，因为这在Python 2.x和3.x之间是不同的）。各种组合都给出相同的结果：如果我不调用 loc_initialize ，程序将在 PyArray_DescrFromType 呼叫。我有NumPy版本1.8.2。作为参考，我在Ubuntu 15.04运行这个。
最令人困惑的是这个C ++ NumPy包装器似乎在不同的翻译单元中调用 import_array b
我缺少什么？为什么必须从同一个翻译单元调用 import_array 才能真正生效？更重要的是，当我从Boost.NumPy包装器之类的不同翻译单元调用 import_array 时，如何让它工作？
通过NumPy头部挖掘，我想我已经找到一个解决方案：
$ c> numpy / __ multiarray_api.h ，有一个部分处理内部API缓冲区应该在哪里。为了简明起见，以下是相关的代码段：
#if defined（PY_ARRAY_UNIQUE_SYMBOL）
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif
#if defined（NO_IMPORT）|| defined（NO_IMPORT_ARRAY）
extern void ** PyArray_API;
#else
#if defined（PY_ARRAY_UNIQUE_SYMBOL）
void ** PyArray_API;
#else
static void ** PyArray_API = NULL;
#endif
#endif
看起来这是为了允许多个模块定义自己的内部API缓冲区，其中每个模块必须调用自己的 import_array define。
要得到几个翻译单元使用相同的内部API缓冲区在每个模块，定义 PY_ARRAY_UNIQUE_SYMBOL 到一些库的唯一名称，然后每个翻译单位定义了import_array包装器定义 NO_IMPORT 或 NO_IMPORT_ARRAY 。顺便提一下，对于ufunc特性也有类似的宏： PY_UFUNC_UNIQUE_SYMBOL 和 NO_IMPORT / NO_IMPORT_UFUNC 。
修改后的工作示例：
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#ifndef MYLIBRARY_USE_IMPORT
#define NO_IMPORT
#endif
#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API
#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API
#include< Python.h>
#include< numpy / npy_3kcompat.h>
#include< numpy / arrayobject.h>
void initialize（）;
#endif
file1.cpp / p>
#define MYLIBRARY_USE_IMPORT
#includeheader1.hpp
void * wrap_import_array
{
import_array（）;
return（void *）1;
}
void initialize（）
{
wrap_import_array（）;
}
file2.cpp
$ b
#includeheader1.hpp
#include< iostream>
int main（）
{
Py_Initialize（）;
initialize（）;
npy_intp dims [] = {5};
std :: cout<< creation descr<< std :: endl;
PyArray_Descr * dtype = PyArray_DescrFromType（NPY_FLOAT64）;
std :: cout<< zeros<< std :: endl;
PyArray_Zeros（1，dims，dtype，0）;
std :: cout<< 清理< std :: endl;
return 0;
}
我不知道这个黑客有什么陷阱，任何更好的替代品，但这似乎至少编译和运行没有任何segfaults。
I'm having problems getting the NumPy C API to properly initialize. I think I've isolated the problem to calling import_array from a different translation unit, but I don't know why this should matter.
Minimal working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
void* loc_wrap_import_array()
{
import_array();
return (void*) 1;
}
void loc_initialize()
{
loc_wrap_import_array();
}
int main()
{
Py_Initialize();
#ifdef USE_LOC_INIT
loc_initialize();
#else
initialize();
#endif
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
Compiler commands:
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m -DUSE_LOC_INIT
./segissue
# runs fine
g++ file1.cpp file2.cpp -o segissue -lpython3.4m -I/usr/include/python3.4m
./segissue
# segfaults
I've tested this with Clang 3.6.0, GCC 4.9.2, Python 2.7, and Python 3.4 (with a suitably modified wrap_import_array because this is different between Python 2.x and 3.x). The various combinations all give the same result: if I don't call loc_initialize, the program will segfault in the PyArray_DescrFromType call. I have NumPy version 1.8.2. For reference, I'm running this in Ubuntu 15.04.
What baffles me most of all is this C++ NumPy wrapper appears to get away with calling import_array in a different translation unit.
What am I missing? Why must I call import_array from the same translation unit in order for it to actually take effect? More importantly, how do I get it to work when I call import_array from a different translation unit like the Boost.NumPy wrapper does?
After digging through the NumPy headers, I think I've found a solution:
in numpy/__multiarray_api.h, there's a section dealing with where an internal API buffer should be. For conciseness, here's the relevant snippet:
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
#define PyArray_API PY_ARRAY_UNIQUE_SYMBOL
#endif
#if defined(NO_IMPORT) || defined(NO_IMPORT_ARRAY)
extern void **PyArray_API;
#else
#if defined(PY_ARRAY_UNIQUE_SYMBOL)
void **PyArray_API;
#else
static void **PyArray_API=NULL;
#endif
#endif
It looks like this is intended to allow multiple modules define their own internal API buffer, in which each module must call their own import_array define.
A consistent way to get several translation units to use the same internal API buffer is in every module, define PY_ARRAY_UNIQUE_SYMBOL to some library unique name, then every translation unit other than the one where the import_array wrapper is defined defines NO_IMPORT or NO_IMPORT_ARRAY. Incidentally, there are similar macros for the ufunc features: PY_UFUNC_UNIQUE_SYMBOL, and NO_IMPORT/NO_IMPORT_UFUNC.
The modified working example:
header1.hpp
#ifndef HEADER1_HPP
#define HEADER1_HPP
#ifndef MYLIBRARY_USE_IMPORT
#define NO_IMPORT
#endif
#define PY_ARRAY_UNIQUE_SYMBOL MYLIBRARY_ARRAY_API
#define PY_UFUNC_UNIQUE_SYMBOL MYLIBRARY_UFUNC_API
#include <Python.h>
#include <numpy/npy_3kcompat.h>
#include <numpy/arrayobject.h>
void initialize();
#endif
file1.cpp
#define MYLIBRARY_USE_IMPORT
#include "header1.hpp"
void* wrap_import_array()
{
import_array();
return (void*) 1;
}
void initialize()
{
wrap_import_array();
}
file2.cpp
#include "header1.hpp"
#include <iostream>
int main()
{
Py_Initialize();
initialize();
npy_intp dims[] = {5};
std::cout << "creating descr" << std::endl;
PyArray_Descr* dtype = PyArray_DescrFromType(NPY_FLOAT64);
std::cout << "zeros" << std::endl;
PyArray_Zeros(1, dims, dtype, 0);
std::cout << "cleanup" << std::endl;
return 0;
}
I don't know what pitfalls there are with this hack or if there are any better alternatives, but this appears to at least compile and run without any segfaults.
这篇关于Segfault时import_array不在同一个翻译单元的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
