分割错误-python C-extension中的核心转储 [英] segmentation fault - core dump in python C-extension
问题描述
我正在为python编写c扩展名.如下所示，该代码的目的是计算两个向量的欧几里得距离. 第一个参数n是向量的维数， 第二个参数，第三个参数是float的两个列表.
I am writing a c-extension for python. As you can see below, the aim of the code is to calculate the euclidean-dist of two vectors. the first param n is the dimension of the vectors, the second , the third param is the two list of float.
我在python中这样调用函数:
I call the function in python like this:
import cutil
cutil.c_euclidean_dist(2,[1.0,1,0],[0,0])
，它运行良好，返回正确的结果. 但是，如果我这样做超过100次(尺寸为1 * 1000)，则会导致分段错误-核心转储:
and it worked well, return the right result. but if i do it for more than 100 times(the dimension is 1*1000), it will cause segmentation fault - core dump:
#!/usr/bin/env python
#coding:utf-8
import cutil
import science
import time
a = []
b = []
d = 0.0
for x in range(2500):
a.append([float(i+x) for i in range(1000)])
b.append([float(i-x) for i in range(1000)])
t1 = time.time()
for x in range(500):
d += cutil.c_euclidean_dist(1000,a[x],b[x])
print time.time() - t1
print d
C代码在这里:
#include <python2.7/Python.h>
#include <math.h>
static PyObject* cutil_euclidean_dist(PyObject* self, PyObject* args) {
PyObject *seq_a, *seq_b;
int n;
float * array_a,* array_b;
PyObject *item;
PyArg_ParseTuple(args,"iOO", &n , &seq_a, &seq_b);
if (!PySequence_Check(seq_a) || !PySequence_Check(seq_b)) {
PyErr_SetString(PyExc_TypeError, "expected sequence");
return NULL;
}
array_a =(float *)malloc(sizeof(float)*n);
array_b =(float *)malloc(sizeof(float)*n);
if (NULL == array_a || NULL == array_b){
PyErr_SetString(PyExc_TypeError, "malloc failed!");
Py_DECREF(seq_a);
Py_DECREF(seq_b);
return NULL;
}
int i;
for(i=0;i<n;i++){
item = PySequence_GetItem(seq_a,i);
if (!PyFloat_Check(item)) {
free(array_a); /* free up the memory before leaving */
free(array_b);
Py_DECREF(seq_a);
Py_DECREF(seq_b);
Py_DECREF(item);
PyErr_SetString(PyExc_TypeError, "expected sequence of float");
return NULL;
}
array_a[i] = PyFloat_AsDouble(item);
Py_DECREF(item);
item = PySequence_GetItem(seq_b,i);
if(!PyFloat_Check(item)) {
free(array_a);
free(array_b);
Py_DECREF(seq_a);
Py_DECREF(seq_b);
Py_DECREF(item);
PyErr_SetString(PyExc_TypeError, "expected sequence of float");
return NULL;
}
array_b[i] = PyFloat_AsDouble(item);
Py_DECREF(item);
}
double sum = 0;
for(i=0;i<n;i++){
double delta = array_a[i] - array_b[i];
sum += delta * delta;
}
free(array_a);
free(array_b);
Py_DECREF(seq_a);
Py_DECREF(seq_b);
return Py_BuildValue("d",sqrt(sum));
}
static PyMethodDef cutil_methods[] = {
{"c_euclidean_dist",(PyCFunction)cutil_euclidean_dist,METH_VARARGS,NULL},
{NULL,NULL,0,NULL}
};
PyMODINIT_FUNC initcutil(void) {
Py_InitModule3("cutil", cutil_methods, "liurui's c extension for python");
}
错误消息:
segmentation fault - core dump:
C扩展名被编译为cutil.so，我不知道如何查看转储. 但是我浏览了我的C代码很多次了，找不到任何问题.
The c-extension is compiled to cutil.so, I do not know how to see the dump. But i looked through my C code for many times,and can not find any problem..
可能是<强>内存问题吗?
这应该是一段非常简单的C代码，这怎么了? 我需要您的帮助〜非常感谢！
It should be a very simple piece of C code, what's wrong with it? I need your help~ thanks a lot !
这是gdb/usr/bin/python2.7 ./core的结果:
here is the result of gdb /usr/bin/python2.7 ./core:
root@ubuntu:/home/rrg/workspace/opencvTest/test# gdb /usr/bin/python2.7 ./core
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/bin/python2.7...Reading symbols from /usr/lib/debug//usr/bin/python2.7...done.
done.
warning: core file may not match specified executable file.
[New LWP 13787]
[New LWP 13789]
[New LWP 13790]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `python py.py'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
309 ../Objects/listobject.c: no such file or directory
#0 0x00000000005398b3 in list_dealloc.16846 (op=0x7f688b2faa28) at ../Objects/listobject.c:309
#1 0x00000000004fdb96 in insertdict_by_entry (value=<optimized out>, ep=0x1777fa8, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:519
#2 insertdict (value=<optimized out>, hash=<optimized out>, key='b', mp=0x7f68a8dbb168) at ../Objects/dictobject.c:556
#3 dict_set_item_by_hash_or_entry (value=<optimized out>, ep=0x0, hash=<optimized out>, key='b',
op={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}) at ../Objects/dictobject.c:765
#4 PyDict_SetItem (
op=op@entry={'a': None, 'x': None, 'c': None, 'b': None, 'd': <float at remote 0x4480b30>, '__builtins__': <module at remote 0x7f68a8de6b08>, 'science': <module at remote 0x7f68a8ce4088>, '__package__': None, 'i': 999, 'cutil': <module at remote 0x7f68a8cdfbb0>, 'time': <module at remote 0x7f68a640ea28>, '__name__': '__main__', 't1': <float at remote 0xd012708>, '__doc__': None}, key=key@entry='b',
value=<optimized out>) at ../Objects/dictobject.c:818
#5 0x000000000055a9e1 in _PyModule_Clear (m=<optimized out>) at ../Objects/moduleobject.c:139
#6 0x00000000004f2ad4 in PyImport_Cleanup () at ../Python/import.c:473
#7 0x000000000042fa89 in Py_Finalize () at ../Python/pythonrun.c:459
#8 0x000000000046ac10 in Py_Main (argc=<optimized out>, argv=0x7fff3958d058) at ../Modules/main.c:665
#9 0x00007f68a8665ec5 in __libc_start_main (main=0x46ac3f <main>, argc=2, argv=0x7fff3958d058, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fff3958d048)
at libc-start.c:287
#10 0x000000000057497e in _start ()
我在最后一个返回值附近评论了最后2个句子:
I comment the last 2 sentences near the last return:
Py_DECREF(seq_a);
Py_DECREF(seq_b);
，然后似乎运作良好.我感到非常奇怪... 这两个句子的目的是释放(或释放)两个pyobject，为什么在没有我认为必要的两个句子的情况下也能很好地工作?
and then it seems to work well. I feel very very strange... The purpose of the two sentence is to free(or release) the two pyobject, why it works well without the two sentences which I think are necessary?
推荐答案
C扩展名被编译为cutil.so，我不知道如何查看转储.
The c-extension is compiled to cutil.so, I do not know how to see the dump.
为解决此问题，我将引用 GNU Radio的GDB/Python调试微型-教程:
To solve this, I'm going to cite GNU Radio's GDB/Python debugging mini-tutorial:
幸运的是，有一个称为核心转储的功能，该功能可以将程序的状态存储在文件中，以便以后进行分析.通常，该功能被禁用；您可以通过以下方式启用它:
Luckily, there's a feature called core dumping that allows the state of your program to be stored in a file, allowing later analysis. Usually, that feature is disabled; you can enable it by:
ulimit -c unlimited
请注意，这仅适用于使用ulimit的shell产生的进程.这里发生的是，核心转储的最大大小设置为无限制(在大多数情况下，原始值为0).
Note that this only works for processes spawned from the shell that you used ulimit in. What happens here is that the maximum size of a core dump is set to unlimited (the original value is 0 in most cases).
现在，核心转储文件位于崩溃的程序的当前执行目录中.在我们的例子中，这是build/python/，但是由于所有核心转储都应具有类似于core.的名称，因此我们可以使用一些查找魔术:
Now, the core dump file lays in the current execution directory of the program that crashed. In our case, that's build/python/, but since all core dumps should have a name like core., we can use a little find magic:
marcus> find -type f -cmin 5 -name 'core.[0-9]*'
./build/python/core.22608
./build/python/core.22608
因为它将找到在最后_5分钟内更改/创建的所有_f_ile，并且它们的名称匹配.
because that will find all _f_iles, changed/created within the last _5 min_utes, having a name that matches.
找到build/python/core.22608， 我们现在可以启动GDB:
having found build/python/core.22608, we can now launch GDB:
gdb programname coredump
即
gdb /usr/bin/python2 build/python/core.22608
很多信息可能会滚动.
最后，您会受到GDB提示的欢迎:
At the end, you're greeted by the GDB prompt:
(gdb)
获取回溯
通常情况下，您只会得到回溯(或更短的时间是bt).回溯只是被调用的函数的层次结构.
Getting a backtrace
Typically, you'd just get a backtrace (or shorter, bt). A backtrace is simply the hierarchy of functions that were called.
(gdb)bt
[...]已跳过
第2帧及其后续代码肯定看起来像是Python实现的一部分-听起来很糟糕，因为GDB本身并不知道如何调试python，但是幸运的是，有一个扩展可以做到这一点.因此，我们可以尝试使用py-bt:
Frame #2 and following definitely look like they're part of the Python implementation -- that sounds bad, because GDB doesn't itself know how to debug python, but luckily, there's an extension to do that. So we can try to use py-bt:
(gdb) py-bt
如果遇到未定义的命令错误，我们必须在此处停止并确保已安装python开发包(Redhatoids上为python-devel，Debianoids上为python2.7-dev)；对于某些系统，应将/usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz]的内容附加到〜/.gdbinit中，然后重新启动gdb.
If we get a undefined command error, we must stop here and make sure that the python development package is installed (python-devel on Redhatoids, python2.7-dev on Debianoids); for some systems, you should append the content of /usr/share/doc/{python-devel,python2.7-dev}/gdbinit[.gz] to your ~/.gdbinit, and re-start gdb.
py-bt的输出现在可以清楚地说明哪些python行对应于哪个堆栈框架(跳过那些对python隐藏的堆栈框架，因为它们在外部库或python实现例程中)
The output of py-bt now states clearly which python lines correspond to which stack frame (skipping those stack frames that are hidden to python, because they are in external libraries or python-implementation routines)
...
这篇关于分割错误-python C-extension中的核心转储的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
