如何创建使用Python C API发电机/迭代器？ [英] How to create a generator/iterator with the Python C API?
问题描述
我如何复制下面的Python code使用Python C API？
How do I replicate the following Python code with the Python C API?
class Sequence():
def __init__(self, max):
self.max = max
def data(self):
i = 0
while i < self.max:
yield i
i += 1
到目前为止，我有这样的：
So far, I have this:
#include <Python/Python.h>
#include <Python/structmember.h>
/* Define a new object class, Sequence. */
typedef struct {
PyObject_HEAD
size_t max;
} SequenceObject;
/* Instance variables */
static PyMemberDef Sequence_members[] = {
{"max", T_UINT, offsetof(SequenceObject, max), 0, NULL},
{NULL} /* Sentinel */
};
static int Sequence_Init(SequenceObject *self, PyObject *args, PyObject *kwds)
{
if (!PyArg_ParseTuple(args, "k", &(self->max))) {
return -1;
}
return 0;
}
static PyObject *Sequence_data(SequenceObject *self, PyObject *args);
/* Methods */
static PyMethodDef Sequence_methods[] = {
{"data", (PyCFunction)Sequence_data, METH_NOARGS,
"sequence.data() -> iterator object\n"
"Returns iterator of range [0, sequence.max)."},
{NULL} /* Sentinel */
};
/* Define new object type */
PyTypeObject Sequence_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Sequence", /* tp_name */
sizeof(SequenceObject), /* tp_basicsize */
0, /* tp_itemsize */
0, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
"Test generator object", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
Sequence_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc)Sequence_init, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
static PyObject *Sequence_data(SequenceObject *self, PyObject *args)
{
/* Now what? */
}
但我不知道下一步去哪里。任何人都可以提供一些建议吗？
But I'm not sure where to go next. Could anyone offer some suggestions?
我想我与这个具有主要问题是模拟收益语句。据我所知，这是一个pretty看起来简单，实则复杂，语句 - 它创建了自己的发电机 __ ITER __（）和的next（），被自动调用的方法。通过文档搜索，似乎要与 PyGenObject 有关;然而，如何创建该对象的一个新实例还不清楚。 PyGen_New（）作为它的参数的 PyFrameObject ，对此我能找到的唯一参考的 PyEval_GetFrame（） ，这似乎并没有成为我想要的（还是我弄错了？）。没有人有这方面的经验，他们可以共享？
I suppose the main problem I'm having with this is simulating the yield statement. As I understand it, it is a pretty simple-looking, but in reality complex, statement — it creates a generator with its own __iter__() and next() methods which are called automatically. Searching through the docs, it seems to be associated with the PyGenObject; however, how to create a new instance of this object is unclear. PyGen_New() takes as its argument a PyFrameObject, the only reference to which I can find is PyEval_GetFrame(), which doesn't seem to be what I want (or am I mistaken?). Does anyone have any experience with this they can share?
我发现我（基本上）扩大Python的东西在做幕后时，这是更清晰的：
I found this to be clearer when I (essentially) expanded what Python was doing behind the scenes:
class IterObject():
def __init__(self, max):
self.max = max
def __iter__(self):
self.i = 0
return self
def next(self):
if self.i >= self.max:
raise StopIteration
self.i += 1
return self.i
class Sequence():
def __init__(self, max):
self.max = max
def data(self):
return IterObject(self.max)
技术上的顺序是关闭的人，但你的想法。
Technically the sequence is off by one but you get the idea.
这个唯一的问题是它是非常恼人的创建一个新的对象，每次一个人需要一台发电机 - 更是如此在Python比C，因为自带的定义新类型所需的怪物的。而且不能有任何收益语句在C，因为C有没有关闭。我做了什么，而不是（因为Python的API中我无法找到它 - 请的指向我一个标准的对象，如果它已经存在！）的创建叫回来一个简单，通用发电机对象类每的next（）方法调用C函数。这是（请注意，我还没有尝试过这种编译，因为它是不完整的 - 见下文）：
The only problem with this is it's very annoying to create a new object every time one needs a generator — even more so in Python than C because of the required monstrosity that comes with defining a new type. And there can be no yield statement in C because C has no closures. What I did instead (since I couldn't find it in the Python API — please point me to a standard object if it already exists!) was create a simple, generic generator object class that called back a C function for every next() method call. Here it is (note that I have not yet tried compiling this because it is not complete — see below):
#include <Python/Python.h>
#include <Python/structmember.h>
#include <stdlib.h>
/* A convenient, generic generator object. */
typedef PyObject *(*callback)(PyObject *callee, void *info) PyGeneratorCallback;
typedef struct {
PyObject HEAD
PyGeneratorCallback callback;
PyObject *callee;
void *callbackInfo; /* info to be passed along to callback function. */
bool freeInfo; /* true if |callbackInfo| should be free'()d when object
* dealloc's, false if not. */
} GeneratorObject;
static PyObject *Generator_iter(PyObject *self, PyObject *args)
{
Py_INCREF(self);
return self;
}
static PyObject *Generator_next(PyObject *self, PyObject *args)
{
return self->callback(self->callee, self->callbackInfo);
}
static PyMethodDef Generator_methods[] = {
{"__iter__", (PyCFunction)Generator_iter, METH_NOARGS, NULL},
{"next", (PyCFunction)Generator_next, METH_NOARGS, NULL},
{NULL} /* Sentinel */
};
static void Generator_dealloc(GenericEventObject *self)
{
if (self->freeInfo && self->callbackInfo != NULL) {
free(self->callbackInfo);
}
self->ob_type->tp_free((PyObject *)self);
}
PyTypeObject Generator_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Generator", /* tp_name */
sizeof(GeneratorObject), /* tp_basicsize */
0, /* tp_itemsize */
Generator_dealloc, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
0, /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
0, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
0, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
/* Returns a new generator object with the given callback function
* and arguments. */
PyObject *Generator_New(PyObject *callee, void *info,
bool freeInfo, PyGeneratorCallback callback)
{
GeneratorObject *generator = (GeneratorObject *)_PyObject_New(&Generator_Type);
if (generator == NULL) return NULL;
generator->callee = callee;
generator->info = info;
generator->callback = callback;
self->freeInfo = freeInfo;
return (PyObject *)generator;
}
/* End of Generator definition. */
/* Define a new object class, Sequence. */
typedef struct {
PyObject_HEAD
size_t max;
} SequenceObject;
/* Instance variables */
static PyMemberDef Sequence_members[] = {
{"max", T_UINT, offsetof(SequenceObject, max), 0, NULL},
{NULL} /* Sentinel */
}
static int Sequence_Init(SequenceObject *self, PyObject *args, PyObject *kwds)
{
if (!PyArg_ParseTuple(args, "k", &self->max)) {
return -1;
}
return 0;
}
static PyObject *Sequence_data(SequenceObject *self, PyObject *args);
/* Methods */
static PyMethodDef Sequence_methods[] = {
{"data", (PyCFunction)Sequence_data, METH_NOARGS,
"sequence.data() -> iterator object\n"
"Returns generator of range [0, sequence.max)."},
{NULL} /* Sentinel */
};
/* Define new object type */
PyTypeObject Sequence_Type = {
PyObject_HEAD_INIT(NULL)
0, /* ob_size */
"Sequence", /* tp_name */
sizeof(SequenceObject), /* tp_basicsize */
0, /* tp_itemsize */
0, /* tp_dealloc */
0, /* tp_print */
0, /* tp_getattr */
0, /* tp_setattr */
0, /* tp_compare */
0, /* tp_repr */
0, /* tp_as_number */
0, /* tp_as_sequence */
0, /* tp_as_mapping */
0, /* tp_hash */
0, /* tp_call */
0, /* tp_str */
0, /* tp_getattro */
0, /* tp_setattro */
0, /* tp_as_buffer */
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_flags*/
"Test generator object", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
0, /* tp_iter */
0, /* tp_iternext */
0, /* tp_methods */
Sequence_members, /* tp_members */
0, /* tp_getset */
0, /* tp_base */
0, /* tp_dict */
0, /* tp_descr_get */
0, /* tp_descr_set */
0, /* tp_dictoffset */
(initproc)Sequence_init, /* tp_init */
0, /* tp_alloc */
PyType_GenericNew, /* tp_new */
};
static PyObject *Sequence_data(SequenceObject *self, PyObject *args)
{
size_t *info = malloc(sizeof(size_t));
if (info == NULL) return NULL;
*info = 0;
/* |info| will be free'()d by the returned generator object. */
GeneratorObject *ret = Generator_New(self, info, true,
&Sequence_data_next_callback);
if (ret == NULL) {
free(info); /* Watch out for memory leaks! */
}
return ret;
}
PyObject *Sequence_data_next_callback(PyObject *self, void *info)
{
size_t i = info;
if (i > self->max) {
return NULL; /* TODO: How do I raise StopIteration here? I can't seem to find
* a standard exception. */
} else {
return Py_BuildValue("k", i++);
}
}
然而，不幸的是，我还没有完成。我已经离开了唯一的问题是：我如何提高与C API一个的StopIteration 例外？我似乎无法找到它在标准异常的上市。此外，也许更重要的是，这是解决这个问题的正确方法是什么？
However, unfortunately, I'm still not finished. The only question I have left is: How do I raise a StopIteration exception with the C API? I can't seem to find it listed in the Standard Exceptions. Also, perhaps more importantly, is this the correct way to approach this problem?
感谢任何人，仍然按照此。
Thanks to anyone that's still following this.
推荐答案
下面是一个简单的实现模块的垃圾邮件一个函数 myiter（INT ）返回迭代器：
Below is a simple implementation of module spam with one function myiter(int) returning iterator:
import spam
for i in spam.myiter(10):
print i
打印数字从0到9。
prints numbers from 0 to 9.
这是简单的，然后你的情况，但显示要点：定义与标准的对象 __ ITER __（）和的next（）方法，以及包括提高执行迭代器的行为的StopIteration 在适当的时候。
It is simpler then your case but shows main points: defining object with standard __iter__() and next() methods, and implementing iterator behaviour including raising StopIteration when appropriate.
在你的情况迭代器对象需要持有参考序列（所以你需要释放器的方法为它Py_DECREF它）。
序列本身需要实施 __国际热核实验堆（）__ 和在里面创建一个迭代器。
In your case iterator object needs to hold reference to Sequence (so you'll need deallocator method for it to Py_DECREF it).
The sequence itself needs to implement __iter()__ and create an iterator inside it.
含有迭代器的状态结构。
（在你的版本，而不是男，那就得参考序列）。
Structure containing state of iterator. (In your version instead of m, it would have reference to Sequence.)
typedef struct {
PyObject_HEAD
long int m;
long int i;
} spam_MyIter;
迭代器 __ __国际热核实验堆（）方法。
它总是简单地返回自。
它允许两个迭代和收集待处理的相同
像构建了...在... 。
PyObject* spam_MyIter_iter(PyObject *self)
{
Py_INCREF(self);
return self;
}
我们的迭代的执行情况：的next（）法
Implementation of our iteration: next() method.
PyObject* spam_MyIter_iternext(PyObject *self)
{
spam_MyIter *p = (spam_MyIter *)self;
if (p->i < p->m) {
PyObject *tmp = Py_BuildValue("l", p->i);
(p->i)++;
return tmp;
} else {
/* Raising of standard StopIteration exception with empty value. */
PyErr_SetNone(PyExc_StopIteration);
return NULL;
}
}
我们需要 PyTypeObject 的扩展版本结构，提供用Python
关于 __ __ ITER信息（）和的next（）。
我们希望他们能够有效地叫，所以没有基于名称的查找词典。
We need extended version of PyTypeObject structure to provide Python with
information about __iter__() and next().
We want them to be called efficiently, so no name-based lookup in dictionary.
static PyTypeObject spam_MyIterType = {
PyObject_HEAD_INIT(NULL)
0, /*ob_size*/
"spam._MyIter", /*tp_name*/
sizeof(spam_MyIter), /*tp_basicsize*/
0, /*tp_itemsize*/
0, /*tp_dealloc*/
0, /*tp_print*/
0, /*tp_getattr*/
0, /*tp_setattr*/
0, /*tp_compare*/
0, /*tp_repr*/
0, /*tp_as_number*/
0, /*tp_as_sequence*/
0, /*tp_as_mapping*/
0, /*tp_hash */
0, /*tp_call*/
0, /*tp_str*/
0, /*tp_getattro*/
0, /*tp_setattro*/
0, /*tp_as_buffer*/
Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_ITER,
/* tp_flags: Py_TPFLAGS_HAVE_ITER tells python to
use tp_iter and tp_iternext fields. */
"Internal myiter iterator object.", /* tp_doc */
0, /* tp_traverse */
0, /* tp_clear */
0, /* tp_richcompare */
0, /* tp_weaklistoffset */
spam_MyIter_iter, /* tp_iter: __iter__() method */
spam_MyIter_iternext /* tp_iternext: next() method */
};
myiter（INT）函数创建迭代器。
static PyObject *
spam_myiter(PyObject *self, PyObject *args)
{
long int m;
spam_MyIter *p;
if (!PyArg_ParseTuple(args, "l", &m)) return NULL;
/* I don't need python callable __init__() method for this iterator,
so I'll simply allocate it as PyObject and initialize it by hand. */
p = PyObject_New(spam_MyIter, &spam_MyIterType);
if (!p) return NULL;
/* I'm not sure if it's strictly necessary. */
if (!PyObject_Init((PyObject *)p, &spam_MyIterType)) {
Py_DECREF(p);
return NULL;
}
p->m = m;
p->i = 0;
return (PyObject *)p;
}
剩下的就是pretty无聊...
The rest is pretty boring...
static PyMethodDef SpamMethods[] = {
{"myiter", spam_myiter, METH_VARARGS, "Iterate from i=0 while i<m."},
{NULL, NULL, 0, NULL} /* Sentinel */
};
PyMODINIT_FUNC
initspam(void)
{
PyObject* m;
spam_MyIterType.tp_new = PyType_GenericNew;
if (PyType_Ready(&spam_MyIterType) < 0) return;
m = Py_InitModule("spam", SpamMethods);
Py_INCREF(&spam_MyIterType);
PyModule_AddObject(m, "_MyIter", (PyObject *)&spam_MyIterType);
}
这篇关于如何创建使用Python C API发电机/迭代器？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
