Python CFFI内存管理问题 [英] Python CFFI memory management issues

问题描述

我正在使用Python 2.7.3在Ubuntu上进行编程.

I am programming on Ubuntu, with Python 2.7.3.

我正在使用CFFI用来自某些C代码的值填充Python列表.
这个列表很大:打印时大约有71000个字符.

I am using CFFI to populate a Python list with values coming from some C code.
That list is quite big : around 71 000 characters long when printed.

C代码正在使用许多库.因此，下面的代码仅是为了更好地了解正在发生的事情.

The C code is using many libraries. Hence, the following code is only here for a better understanding of what is happening.

datas_list = []
for i in range( 0, x ):
    c_pDataStructure = ffi.new( "c_DataStructure[]", 1 )    // Create a pointer to the data structure
    c.SomeCFunction( c_pDataStructure )    // Populate the data structure
    datas_list.append( c.GetSomeInfo( c_pDataStructure ) )    // Get some info from the data structure
    c.FreeDataStructure( c_pDataStructure )    // Release dynamically allocated memory

该程序在Wingware IDE中运行良好，但在以下位置从命令行启动时以glibc错误(*** glibc detected *** python: free(): invalid next size (fast): 0x0000000003b0b080 ***)结尾:

The program runs well using Wingware IDE but ends with a glibc error (*** glibc detected *** python: free(): invalid next size (fast): 0x0000000003b0b080 ***) when started from the command line, right before:

c_pDataStructure = ffi.new( "c_Datastructure[]", 1)  

阅读了wim的 answer ，我检查了IDE和命令行是否都使用相同的解释器来运行代码-它们是(/usr/bin/python).

After reading wim's answer, I checked if both the IDE and the command line were running the code using the same interpreter — they are (/usr/bin/python).

编辑(Valgrind报告):

==5089== Process terminating with default action of signal 11 (SIGSEGV)  
==5089==  General Protection Fault  
==5089==    at 0x54FBB0: PyObject_Malloc (in /usr/bin/python2.7)  
==5089==    by 0x10B30625: allocate_owning_object (_cffi_backend.c:2972)  
==5089==    by 0x10B40EE8: allocate_with_allocator.constprop.84 (_cffi_backend.c:3032)  
==5089==    by 0x10B41010: direct_newp (_cffi_backend.c:3153)  
==5089==    by 0x10B4138C: b_newp (_cffi_backend.c:3177)  
==5089==    by 0x4F95A4: PyEval_EvalFrameEx (in /usr/bin/python2.7)  
==5089==    by 0x5008C1: PyEval_EvalCodeEx (in /usr/bin/python2.7)  
==5089==    by 0x4F9AB7: PyEval_EvalFrameEx (in /usr/bin/python2.7)  
==5089==    by 0x4F9D01: PyEval_EvalFrameEx (in /usr/bin/python2.7)  
==5089==    by 0x4F9D01: PyEval_EvalFrameEx (in /usr/bin/python2.7)  
==5089==    by 0x4F9D01: PyEval_EvalFrameEx (in /usr/bin/python2.7)  
==5089==    by 0x4F9D01: PyEval_EvalFrameEx (in /usr/bin/python2.7)

这里是有关C数据结构的更多信息.看起来就是这样:

Here are some more information about the C data structure. This is how it looks :

typedef struct _STRUCT3{
    some int, char*
}STRUCT3, *PSTRUCT3;

typedef struct _STRUCT2{
    some int
    PSTRUCT3 pStruct3;
}STRUCT3, *PSTRUCT3;

typedef struct _STRUCT1{
    some int, char*
    PSTRUCT2 pStruct2;
}STRUCT1, *PSTRUCT1;

我制作了一个小C程序来分配/取消分配完整的C结构，并且valgrind没有发现任何内存泄漏.

I made a little C program to allocate / deallocate a full C structure and valgrind did not find any memory leak.

问题:

• 以上valgrind报告的确切含义是什么?
• 从 IDE和从命令行?
  注意:IDE使用Python参数-u (unbuffered)来运行程序，但是将其添加到命令行中并没有区别.
• 当我自己取消分配结构时，Python的垃圾收集器在起作用吗?我应该改用ffi.gc( c_pDataStructure, c.FreeDataStructure )吗?

• What does the above valgrind report exactly mean?
• What could be the differences between running the program from the IDE and from the command line?
  Note: The IDE uses the Python argument -u (unbuffered) to run the program, but adding it to the command line makes no difference.
• As I deallocate the structure on my own, is Python's garbage collector acting? Should I use ffi.gc( c_pDataStructure, c.FreeDataStructure ) instead?

推荐答案

我找到了解决问题的方法:

I found how to fix my issues:

我用ffi.gc(cdata, destructor)创建了结构.我的Python代码现在看起来像:

I used ffi.gc(cdata, destructor) to create the structure. My Python code now looks like:

data_list = []
for i in range( 0, x ):  
    # Create a pointer to the data structure and tell the garbage collector how to destroy it
    gc_c_pDataStructure = ffi.gc( c.CreateDataStructure(), c.FreeDataStructure )
    c.SomeCFunction( gc_c_pDataStructure )  # Populate the data structure
    datas_list.append( c.GetSomeInfo( gc_c_pDataStructure ) ) # Store some data  

以下是与ffi.gc()相关的链接:

• Python CFFI文档
• 大卫的线程

• Python CFFI documentation
• David's thread

这是C函数，用于创建数据结构(根据问题中的结构示例):

And here is the C function to create the data structure (according to the structure example from the question):

 PSTRUCT1 CreateDataStructure()
 {
     PSTRUCT1 pStruct1 = ( PSTRUCT1 ) malloc( sizeof( STRUCT1 ) );
     _SetDummyValues( pStruct1 );

     return pStruct1;
 }

如您所见，我必须创建函数void _SetDummyValues( PSTRUCT1 pStruct1 ).该函数将给定的结构指针设置为NULL.

As you can see, I had to create the function void _SetDummyValues( PSTRUCT1 pStruct1 ). That function sets the given structure pointers to NULL.

这篇关于Python CFFI内存管理问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！