使用Py_buffer和PyMemoryView_FromBuffer具有不同的项目大小 [英] Using Py_buffer and PyMemoryView_FromBuffer with different itemsizes

问题描述

这个问题与我之前问过的问题有关.即这一个任何人都有兴趣.基本上，我想做的是使用包裹在memoryview对象中的Py_buffer将C数组公开给Python.我已经使用PyBuffer_FillInfo使它正常工作(工作=我可以在Python中操作数据并将其写入C中的stdout)，但是如果我尝试滚动自己的缓冲区，则在之后会出现段错误 C函数返回.

This question is related to a previous question I asked. Namely this one if anyone is interested. Basically, what I want to do is to expose a C array to Python using a Py_buffer wrapped in a memoryview-object. I've gotten it to work using PyBuffer_FillInfo (work = I can manipulate the data in Python and write it to stdout in C), but if I try to roll my own buffer I get a segfault after the C function returns.

我需要创建自己的缓冲区，因为PyBuffer_FillInfo假定格式为char，因此将itemize字段设置为1.我需要能够提供大小为1、2、4和8的项目.

I need to create my own buffer because PyBuffer_FillInfo assumes that the format is char, making the itemsize field 1. I need to be able to provide items of size 1, 2, 4 and 8.

一些代码，这是一个有效的示例:

Some code, this is a working example:

Py_buffer *buf = (Py_buffer *) malloc(sizeof(*buf));
int r = PyBuffer_FillInfo(buf, NULL, malloc(sizeof(char) * 4), 4, 0, PyBUF_CONTIG);
PyObject *mv = PyMemoryView_FromBuffer(buf);
//Pack the memoryview object into an argument list and call the Python function
for (blah)
  printf("%c\n", *buf->buf++); //this prints the values i set in the Python function

看PyBuffer_FillInfo的实现，这很简单，我滚动了自己的函数以能够提供自定义项目大小:

Looking at the implementation of PyBuffer_FillInfo, which is really simple, I rolled my own function to be able to provide custom itemsizes:

//buffer creation function
Py_buffer *getReadWriteBuffer(int nitems, int itemsize, char *fmt) {
  Py_buffer *buf = (Py_buffer *) malloc(sizeof(*buf));
  buf->obj = NULL
  buf->buf = malloc(nitems * itemsize);
  buf->len = nitems * itemsize;
  buf->readonly = 0;
  buf->itemsize = itemsize;
  buf->format = fmt;
  buf->ndim = 1;
  buf->shape = NULL;
  buf->strides = NULL;
  buf->suboffsets = NULL;
  buf->internal = NULL;
  return buf;
}

我如何使用它:

Py_buffer *buf = getReadWriteBuffer(32, 2, "h");
PyObject *mv = PyMemoryView_FromBuffer(buf);
// pack the memoryview into an argument list and call the Python function as before

for (blah)
  printf("%d\n", *buf->buf); //this prints all zeroes even though i modify the array in Python

return 0;
//the segfault happens somewhere after here

使用我自己的缓冲区对象的结果是C函数返回后出现了段错误.我真的不明白为什么会这样.任何帮助将不胜感激.

The result of using my own buffer object is a segfault after the C function returns. I really don't understand why this happens at all. Any help would be most appreciated.

编辑 根据这个问题，其中我以前找不到，itemsize> 1甚至可能根本不被支持.这使这个问题更加有趣.也许我可以在具有足够大内存的地方使用PyBuffer_FillInfo来保存我想要的内容(例如32 C浮点数).在这种情况下，问题更多是关于如何将Python浮点数分配给Python函数中的memoryview对象.问问题.

EDIT According to this question, which I failed to find before, itemsize > 1 might not even be supported at all. Which makes this question even more interesting. Maybe I could use PyBuffer_FillInfo with a large enough block of memory to hold what I want (32 C floats for example). In that case, the question is more about how to assign Python floats to the memoryview object in the Python function. Questions questions.

推荐答案

因此，在没有答案的情况下，我决定采用不同于我最初打算的另一种方法.留在这里，以防其他人碰到同样的障碍.

So, in lack of answers I decided to take another approach than the one I originally intended. Leaving this here in case someone else hits the same snag.

基本上，与其在C中创建一个缓冲区(或等价的字节数组)，然后将其传递给Python以供扩展用户修改.我只是稍微重新设计了一下代码，以便用户从Python回调函数返回一个字节数组(或支持缓冲区接口的任何类型).这样，我什至不必担心项目的大小，因为就我而言，所有C代码对返回的对象所做的工作都是提取其缓冲区，然后使用简单的memcpy复制到另一个缓冲区.

Basically, instead of creating a buffer (or bytearray, equiv.) in C and passing it to Python for the extension user to modify. I simply redesigned the code slightly, so that the user returns a bytearray (or any type that supports the buffer interface) from the Python callback function. This way I need not even worry about the size of the items since, in my case, all the C code does with the returned object is to extract its buffer and copy it to another buffer with a simple memcpy.

代码:

PYGILSTATE_ACQUIRE; //a macro i made
PyObject *result = PyEval_CallObject(python_callback, NULL);
if (!PyObject_CheckBuffer(result))
  ; //raise exception

Py_buffer *view = (Py_buffer *) malloc(sizeof(*view));
int error = PyObject_GetBuffer(result, view, PyBUF_SIMPLE);
if (error)
  ; //raise exception

memcpy(my_other_buffer, view->buf, view->len);

PyBuffer_Release(view);
Py_DECREF(result);
PYGILSTATE_RELEASE; //another macro

我希望这对某人有帮助.

I hope this helps someone.

这篇关于使用Py_buffer和PyMemoryView_FromBuffer具有不同的项目大小的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！