我可以强制numpy的ndarray采取其内存的所有权？ [英] Can I force a numpy ndarray to take ownership of its memory?

问题描述

我有一个C函数mallocs（）和填充彩车的二维数组。它返回该地址和数组的大小。签名是

I have a C function that mallocs() and populates a 2D array of floats. It "returns" that address and the size of the array. The signature is

int get_array_c(float** addr, int* nrows, int* ncols);

我想从Python中调用它，所以我用ctypes的。

I want to call it from Python, so I use ctypes.

import ctypes
mylib = ctypes.cdll.LoadLibrary('mylib.so')
get_array_c = mylib.get_array_c

我从来没有想出如何指定参数类型与ctypes的。我往往只写我使用每个C函数Python包装，并确保我得到的正确类型的包装。花车的阵列在列主顺序一个矩阵，我想获得它作为一个numpy.ndarray。但它的pretty很大，所以我想使用C函数分配的内存，而不是复制。 （我只是发现这个东西PyBuffer_FromMemory在这个StackOverflow的答案： http://stackoverflow.com/a/4355701/3691 ）

buffer_from_memory = ctypes.pythonapi.PyBuffer_FromMemory
buffer_from_memory.restype = ctypes.py_object

import numpy
def get_array_py():
    nrows = ctypes.c_int()
    ncols = ctypes.c_int()
    addr_ptr = ctypes.POINTER(ctypes.c_float)()
    get_array_c(ctypes.byref(addr_ptr), ctypes.byref(nrows), ctypes.byref(ncols))
    buf = buffer_from_memory(addr_ptr, 4 * nrows * ncols)
    return numpy.ndarray((nrows, ncols), dtype=numpy.float32, order='F',
                         buffer=buf)

这似乎给我正确的价值观的数组。但我pretty相信这是一个内存泄漏。

This seems to give me an array with the right values. But I'm pretty sure it's a memory leak.

>>> a = get_array_py()
>>> a.flags.owndata
False

该阵列不拥有的内存。很公平;默认情况下，当从一个缓冲区创建阵列，它不应该。但在这种情况下，它应该。当numpy的数组被删除时，我真的很喜欢蟒蛇来释放我的缓冲存储器中。好像如果我能迫使owndata为True，应该这样做，但owndata不可设置。

The array doesn't own the memory. Fair enough; by default, when the array is created from a buffer, it shouldn't. But in this case it should. When the numpy array is deleted, I'd really like python to free the buffer memory for me. It seems like if I could force owndata to True, that should do it, but owndata isn't settable.

解决方案不能令人满意：

Unsatisfactory solutions:

1. 请get_array_py的调用者（）负责释放内存。这是超级讨厌;主叫方应该能够处理这个numpy的阵列，就像任何其他numpy的数组。

1. Make the caller of get_array_py() responsible for freeing the memory. That's super annoying; the caller should be able to treat this numpy array just like any other numpy array.

原始数组复制到在get_array_py新numpy的阵列（有自己的，独立显存），删除第一阵列，​​并释放内get_array_py内存（）。返回，而不是原始数组副本。这是烦人，因为它是一个应该对是不必要的内存拷贝。

Copy the original array into a new numpy array (with its own, separate memory) in get_array_py, delete the first array, and free the memory inside get_array_py(). Return the copy instead of the original array. This is annoying because it's an ought-to-be unnecessary memory copy.

有没有办法做我想做什么？我不能修改的C函数本身，虽然我可以添加其他的C函数库，如果这是有帮助的。

Is there a way to do what I want? I can't modify the C function itself, although I could add another C function to the library if that's helpful.

推荐答案

我只是偶然发现了这个问题，它还是在2013年八月的一个问题numpy的是真的挑剔 OWNDATA 标志：有没有办法可以在Python层面进行修改，所以ctypes的将最有可能无法做到这一点。在numpy的C-API级 - 现在我们正在谈论的Python使得扩展模块的完全不同的方式 - 人们必须明确地设置标志和：

I just stumbled upon this question, which is still an issue in August 2013. Numpy is really picky about the OWNDATA flag: There is no way it can be modified on the Python level, so ctypes will most likely not be able to do this. On the numpy C-API level - and now we are talking about a completely different way of making Python extension modules - one has to explicitly set the flag with:

PyArray_ENABLEFLAGS(arr, NPY_ARRAY_OWNDATA);

在numpy的＆LT; 1.7，人们不得不更加明确的：

On numpy < 1.7, one had to be even more explicit:

((PyArrayObject*)arr)->flags |= NPY_OWNDATA;

如果一个人在底层C函数/库的控制，最好的办法是通过它的Python适当大小的空numpy的阵列来存储结果，基本原则是，内存分配应该总是做上的最高水平可能的，在这种情况下，上了Python间preTER的水平

If one has any control over the underlying C function/library, the best solution is to pass it an empty numpy array of the appropriate size from Python to store the result in. The basic principle is that memory allocation should always be done on the highest level possible, in this case on the level of the Python interpreter.

由于kynan下面评论，如果你使用用Cython ，你必须揭露功能 PyArray_ENABLEFLAGS 手动，请参阅本后<一个href=\"http://stackoverflow.com/questions/23872946/force-numpy-ndarray-to-take-ownership-of-its-memory-in-cython\">Force numpy的ndarray采取它在用Cython 内存的所有权。

As kynan commented below, if you use Cython, you have to expose the function PyArray_ENABLEFLAGS manually, see this post Force NumPy ndarray to take ownership of its memory in Cython.

这篇关于我可以强制numpy的ndarray采取其内存的所有权？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！