Python C包装器内存泄漏 [英] Python C Wrapper Memory Leak

问题描述

我对python和C有一定的经验，但是刚开始编写python模块作为C函数的包装.对于一个项目，我需要一个名为"score"的函数来运行，其运行速度比我在python中获得的运行速度快得多，因此我在C中进行了编码，实际上只是希望能够从python进行调用.它接受一个python整数列表，我希望C函数获取一个整数数组，该数组的长度，然后将一个整数返回给python.这是我当前的(有效的)解决方案.

I am moderately experienced in python and C but new to writing python modules as wrappers on C functions. For a project I needed one function named "score" to run much faster than I was able to get in python so I coded it in C and literally just want to be able to call it from python. It takes in a python list of integers and I want the C function to get an array of integers, the length of that array, and then return an integer back to python. Here is my current (working) solution.

static PyObject *module_score(PyObject *self, PyObject *args) {
    int i, size, value, *gene;
    PyObject *seq, *data;

    /* Parse the input tuple */
    if (!PyArg_ParseTuple(args, "O", &data))
        return NULL;
    seq = PySequence_Fast(data, "expected a sequence");
    size = PySequence_Size(seq);

    gene = (int*) PyMem_Malloc(size * sizeof(int));
    for (i = 0; i < size; i++)
        gene[i] = PyInt_AsLong(PySequence_Fast_GET_ITEM(seq, i));

    /* Call the external C function*/
    value = score(gene, size);

    PyMem_Free(gene);

    /* Build the output tuple */
    PyObject *ret = Py_BuildValue("i", value);
    return ret;
}

这有效，但似乎会泄漏内存，并且以我无法忽略的速度.通过暂时使得分函数仅返回0并仍然看到泄漏行为，我确保了所示函数中发生泄漏.我以为对PyMem_Free的调用应该照顾PyMem_Malloc的存储，但是我目前的猜测是，此函数中的某些内容正在分配并保留在每个调用中，因为泄漏行为与对该函数的调用次数成正比.我是否没有正确执行序列到数组的转换，或者我可能没有效率地返回结束值?感谢您的帮助.

This works but seems to leak memory and at a rate I can't ignore. I made sure that the leak is happening in the shown function by temporarily making the score function just return 0 and still saw the leaking behavior. I had thought that the call to PyMem_Free should take care of the PyMem_Malloc'ed storage but my current guess is that something in this function is getting allocated and retained on each call since the leaking behavior is proportional to the number of calls to this function. Am I not doing the sequence to array conversion correctly or am I possibly returning the ending value inefficiently? Any help is appreciated.

推荐答案

seq是一个新的Python对象，因此您需要删除该对象.您还应该检查seq是否也为NULL.

seq is a new Python object so you will need delete that object. You should check if seq is NULL, too.

(未经测试)之类的东西

Something like (untested):

static PyObject *module_score(PyObject *self, PyObject *args) {
    int i, size, value, *gene;
    long temp;
    PyObject *seq, *data;

    /* Parse the input tuple */
    if (!PyArg_ParseTuple(args, "O", &data))
        return NULL;
    if (!(seq = PySequence_Fast(data, "expected a sequence")))
        return NULL;

    size = PySequence_Size(seq);

    gene = (int*) PyMem_Malloc(size * sizeof(int));
    for (i = 0; i < size; i++) {
        temp = PyInt_AsLong(PySequence_Fast_GET_ITEM(seq, i));
        if (temp == -1 && PyErr_Occurred()) {
            Py_DECREF(seq);
            PyErr_SetString(PyExc_ValueError, "an integer value is required");
            return NULL;
        }
        /* Do whatever you need to verify temp will fit in an int */
        gene[i] = (int*)temp;
    }

    /* Call the external C function*/
    value = score(gene, size);

    PyMem_Free(gene);
    Py_DECREF(seq):

    /* Build the output tuple */
    PyObject *ret = Py_BuildValue("i", value);
    return ret;
}

这篇关于Python C包装器内存泄漏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！