分配给具有项大小>的Python 3.x缓冲区1个 [英] Assignment into Python 3.x Buffers with itemsize > 1

问题描述

我正在尝试通过Python 3.x缓冲区接口公开图像像素信息的缓冲区(32位RGBA).经过大量的玩耍之后，我能够像下面这样工作:

I am trying to expose a buffer of image pixel information (32 bit RGBA) through the Python 3.x buffer interface. After quite a bit of playing around, I was able to get this working like so:

int Image_get_buffer(PyObject* self, Py_buffer* view, int flags)
{
    int img_len;
    void* img_bytes;

    // Do my image fetch magic
    get_image_pixel_data(self, &img_bytes, &img_len);

    // Let python fill my buffer
    PyBuffer_FillInfo(view, self, img_bytes, img_len, 0, flags); 
}

在python中，我可以像这样玩它:

And in python I can play with it like so:

mv = memoryview(image)
print(mv[0]) # prints b'\x00'
mv[0] = b'\xFF' # set the first pixels red component to full
mx[0:4] = b'\xFF\xFF\xFF\xFF' # set the first pixel to white

那很棒.但是，如果我可以使用完整的像素值(int，4个字节)而不是单个字节，那就太好了，因此我修改了缓冲区提取，如下所示:

And that works splendidly. However, it would be great if I could work with the full pixel value (int, 4 byte) instead of individual bytes, so I modified the buffer fetch like so:

int Image_get_buffer(PyObject* self, Py_buffer* view, int flags)
{
    int img_len;
    void* img_bytes;

    // Do my image fetch magic
    get_image_pixel_data(self, &img_bytes, &img_len);

    // Fill my buffer manually (derived from the PyBuffer_FillInfo source)
    Py_INCREF(self);
    view->readonly = 0;
    view->obj = self;
    view->buf = img_bytes;
    view->itemsize = 4;
    view->ndim = 1;
    view->len = img_len;
    view->suboffsets = NULL;

    view->format = NULL;
    if ((flags & PyBUF_FORMAT) == PyBUF_FORMAT)
        view->format = "I";

    view->shape = NULL;
    if ((flags & PyBUF_ND) == PyBUF_ND)
    {
        Py_ssize_t shape[] = { (int)(img_len/4) };
        view->shape = shape;
    }

    view->strides = NULL;
    if((flags & PyBUF_STRIDED) == PyBUF_STRIDED)
    {
        Py_ssize_t strides[] = { 4 };
        view->strides = strides;
    }

    return 0;
}

这实际上返回了数据，我可以正确地读取它，但是现在任何尝试将值分配给它的尝试都失败了！

This actually returns the data and I can read it correctly, but any attempt to assign a value into it now fails!

mv = memoryview(image)
print(mv[0]) # prints b'\x00\x00\x00\x00'
mv[0] = 0xFFFFFFFF # ERROR (1)
mv[0] = b'\xFF\xFF\xFF\xFF' # ERROR! (2)
mv[0] = mv[0] # ERROR?!? (3)

在第一种情况下，该错误通知我'int' does not support the buffer interface，这是一个可耻和令人困惑的问题(我确实指定了缓冲区格式毕竟是"I")，但是我可以解决这个问题.但是，在情况2和3中，情况变得很奇怪:这两种情况都使我遇到TypeError读取mismatching item sizes for "my.Image" and "bytes"(显然，my.Image是我的图像类型)

In case 1 the error informs me that 'int' does not support the buffer interface, which is a shame and a bit confusing (I did specify that the buffer format was "I" after all), but I can deal with that. In case 2 and 3 things get really weird, though: Both cases gime me an TypeError reading mismatching item sizes for "my.Image" and "bytes" (Where my.Image is, obviously, my image type)

这对我来说很困惑，因为我传入的数据大小显然与从该元素中取出的数据大小相同.如果itemsize大于1，似乎缓冲区只是停止允许分配.当然，此接口的文档确实很稀疏，并且在python代码中仔细研究并没有给出任何用法示例，因此我相当棘手.我是否缺少一些说明当itemsize> 1时缓冲区实际上变得无用"的文档摘要，是我做错了我看不到的错误，还是这是Python中的错误? (针对3.1.1进行测试)

This is very confusing to me, since the data I'm passing in is obviously the same size as what I get out of that element. It seems as though buffers simply stop allowing assignment if the itemsize is greater than 1. Of course, the documentation for this interface is really sparse and perusing through the python code doesn't really give any usage examples so I'm fairly stuck. Am I missing some snippit of documentation that states "buffers become essentially useless when itemsize > 1", am I doing something wrong that I can't see, or is this a bug in Python? (Testing against 3.1.1)

感谢您可以就此(公认的高级)问题提供任何见识！

Thanks for any insight you can give on this (admittedly advanced) issue!

推荐答案

我在函数memory_ass_sub中的python代码(在objects的memoryobject.c中)中找到了此代码:

I found this in the python code (in memoryobject.c in Objects) in the function memory_ass_sub:

/* XXX should we allow assignment of different item sizes
   as long as the byte length is the same?
   (e.g. assign 2 shorts to a 4-byte slice) */
if (srcview.itemsize != view->itemsize) {
    PyErr_Format(PyExc_TypeError,
        "mismatching item sizes for \"%.200s\" and \"%.200s\"", 
        view->obj->ob_type->tp_name, srcview.obj->ob_type->tp_name);
    goto _error;
}

这是后两个错误的来源.看起来即使mv [0]的项目大小仍不等于其自身.

that's the source of the latter two errors. It looks like the itemsize for even mv[0] is still not equal to itself.

更新

这就是我想发生的事情.当您尝试在mv中分配某些内容时，它将在Objects/memoryobject.c中调用memory_ass_sub，但是该函数仅将PyObject作为输入.然后使用PyObject_GetBuffer函数将该对象更改为内部的缓冲区，即使在mv [0]的情况下，它已经是一个缓冲区(以及您想要的缓冲区！).我的猜测是，此函数将对象带入一个简单的itemize = 1缓冲区中，而不管它是否已经是一个缓冲区.这就是为什么即使对于

Here's what I think is going on. When you try to assign something in mv, it calls memory_ass_sub in Objects/memoryobject.c, but that function takes only a PyObject as input. This object is then changed into a buffer inside using the PyObject_GetBuffer function even though in the case of mv[0] it is already a buffer (and the buffer you want!). My guess is that this function takes the object and makes it into a simple buffer of itemsize=1 regardless of whether it is already a buffer or not. That is why you get the mismatching item sizes even for

mv[0] = mv[0]

第一次分配的问题

mv [0] = 0xFFFFFFFF

mv[0] = 0xFFFFFFFF

检查我是否可以将int用作缓冲区，这是目前我不了解的.

stems (I think) from checking if the int is able to be used as a buffer, which currently it isn't set-up for from what I understand.

换句话说，缓冲系统当前无法处理大于1的项目.看起来好像还差得很远，但是最终需要做更多的工作.如果您确实可以使用它，则应该将更改提交回主Python发行版.

In other words, the buffer system isn't currently able to handle item sizes bigger from 1. It doesn't look like it is so far off, but it would take a bit more work on your end. If you do get it working, you should probably submit the changes back to the main Python distribution.

另一个更新

您第一次尝试分配mv [0]时的错误代码是由于int在调用PyObject_CheckBuffer时失败PyObject_CheckBuffer引起的.显然，系统仅处理可缓冲对象的副本.似乎也应该更改它.

The error code from your first try at assigning mv[0] stems from the int failing the PyObject_CheckBuffer when PyObject_CheckBuffer is called on it. Apparently the system only handles copies from bufferable objects. This seems like it should be changed too.

结论

目前，Python缓冲区系统无法处理您猜到的itemsize> 1的项目.而且，它无法处理来自不可缓冲对象(例如整数)的缓冲区分配.

Currently the Python buffer system can't handle items with itemsize > 1 as you guessed. Also, it can't handle assignments to a buffer from non-bufferable objects such as ints.

这篇关于分配给具有项大小>的Python 3.x缓冲区1个的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！