PyTuple_SetItem的局限性 [英] Limitations of PyTuple_SetItem

问题描述

我有一个Python扩展模块，该模块创建一个元组作为另一个对象的属性，并在该元组中设置项目.每当我在Python中执行此模块时，都会不断出现错误SystemError: bad argument to internal function

I have a Python extension module which creates a tuple as an attribute of another object, and sets items in the tuple. Whenever I execute this module in Python, I keep getting the error SystemError: bad argument to internal function

在阅读了PyTuple的文档，并调试了几个小时的程序之后，我仍然不知道到底发生了什么.通过调试器运行我的程序，表明问题是在Python解释器内部的库调用中发生的.因此，最后，我查看了Python源代码，终于我意识到了这个问题. PyTuple_SetItem函数有一个有趣的限制，我不知道，也找不到明确记录的文件.

After reading over the docs for PyTuple, and debugging my program for a few hours, I still couldn't figure out what the hell was going on. Running my program through a debugger indicated the problem was occurring within a library call inside the Python interpreter. So, finally, I looked at the Python source code, and at long last I realized the problem. The PyTuple_SetItem function has an interesting restriction which I didn't know about, and can't find explicitly documented.

这是Python源代码中的重要功能(为清晰起见进行了编辑):

Here is the important function in the Python source (edited for clarity):

int PyTuple_SetItem(register PyObject *op, register Py_ssize_t i, PyObject *newitem)
{
    .....
    if (!PyTuple_Check(op) || op->ob_refcnt != 1) {
        Py_XDECREF(newitem);
        PyErr_BadInternalCall();
        return -1;
    }
    .....
}

这里重要的一行是条件 op-> ob_refcnt！= 1 .这就是问题所在:除非元组的引用计数为1，否则您都无法调用PyTuple_SetItem.看起来这里的想法是，除非在创建元组之后，否则您就永远不应该使用PyTuple_SetItem.使用PyTuple_New().我猜这是有道理的，因为毕竟元组应该是不可变的，所以此限制有助于使您的C代码更符合Python类型系统的抽象.

The important line here is the condition op->ob_refcnt != 1. So here's the problem: you can't even call PyTuple_SetItem unless the Tuple has a ref-count of 1. It looks like the idea here is that you're never supposed to use PyTuple_SetItem except right after you create a tuple using PyTuple_New(). I guess this makes sense, since Tuples are supposed to be immutable, after all, so this restriction helps keep your C code more in line with the abstractions of the Python type system.

但是，我在任何地方都找不到此限制.相关文档似乎在此处和此处，两者均未指定此限制.文档基本上说，当您调用PyTuple_New(X)时，元组中的所有项目都将初始化为NULL.由于NULL不是有效的Python值，因此由扩展模块程序员确定在将Tuple返回给解释器之前，必须在Tuple中的所有插槽中都填充正确的Python值.但是它并没有说在Tuple对象的引用计数为1时必须执行此操作.

However, I can't find this restriction documented anywhere. Relevant docs seem to be here and here, neither of which specify this restriction. The docs basically say that when you call PyTuple_New(X), all items in the tuple are initialized to NULL. Since NULL is not a valid Python value, it's up to the extension-module programmer to make sure that all slots in the Tuple are filled in with proper Python values before returning the Tuple to the interpreter. But it doesn't say anywhere that this must be done while the Tuple object has a ref count of 1.

所以现在的问题是，我基本上已经将自己编码到一个角落，因为我不知道对PyTuple_SetItem的此限制(未记录?).我的代码以这样的方式构造:在元组本身成为另一个对象的属性之前，在元组中插入项目非常不方便.因此，当需要填充元组中的项目时，元组已经具有较高的引用计数.

So now, the problem is that I've basically coded myself into a corner because I wasn't aware of this (undocumented?) restriction on PyTuple_SetItem. My code is structured in such a way that it's very inconvenient to insert items into the tuple until after the tuple itself has become an attribute of another object. So, when it comes time to fill in the items in the tuple, the tuple already has a higher ref count.

我可能必须重组我的代码，但是我认真考虑只是将元组上的引用计数临时设置为1，插入项目，然后恢复原始引用计数.当然，我知道这是一个可怕的骇客，不是任何永久解决方案.无论如何，我想知道关于元组上引用计数的要求是否在文档中在任何地方.仅仅是CPython的实现细节，还是API用户可以期望的行为依赖它?

I'll probably have to restructure my code, but I was seriously considering just temporarily setting the ref count on the Tuple to 1, inserting the items, and then restoring the original ref count. Of course, that's a horrible hack, I know, and not any kind of permanent solution. Regardless, I'd like to know if the requirement regarding the ref count on the Tuple is documented anywhere. Is it just an implementation detail of CPython, or is it something that API users can rely on as expected behavior?

推荐答案

我很确定您可以使用PyTuple_SET_ITEM而不是PyTuple_SetItem来解决这些限制. PyTuple_SET_ITEM是在tupleobject.h中定义的宏，如下所示:

I'm quite sure that you can get around the restrictions by using PyTuple_SET_ITEM instead of PyTuple_SetItem. PyTuple_SET_ITEM is a macro defined in tupleobject.h as follows:

#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject*)(op))->ob_item[i] = v

因此，如果您绝对，绝对和完全确定:

So, if you are absolutely, definitely and utterly sure that:

1. op是一个元组对象
2. 到目前为止，您尚未在元组中初始化插槽i
3. 您拥有对v的引用，并且想让元组窃取它，并且
4. 在调用PyTuple_SET_ITEM
之前，另一个Python对象不可能使用元组进行任何操作

1. op is a tuple object
2. you haven't initialized slot i in the tuple so far
3. you own a reference to v and you want to let the tuple steal it and
4. there is no chance of another Python object using the tuple for anything before you call PyTuple_SET_ITEM

那么我想您可以安全使用PyTuple_SET_ITEM.

then I guess you are safe to use PyTuple_SET_ITEM.

这篇关于PyTuple_SetItem的局限性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！