yield如何在Python C代码中正常工作&不良部分 [英] How do yield works in Python C code, good & bad part
问题描述
最近，我一直在研究Python的代码.我知道如何使用生成器(下一步，发送等)，但是通过阅读Python C代码来理解它很有趣.
Recently I've been looking into the code of Python. I know how to use generators (next, send and etc..), but it's fun to understand it by reading the Python C code.
我在 Object/genobject.c 中找到了代码，而且理解起来并不难(但仍然不容易).所以我想知道它是如何工作的，并确保我对Python的生成器没有误解.
I found the code in Object/genobject.c, and it's not that hard (but still not easy) to understand. So I want to know how it really works, and make sure I do not have a misunderstanding about generators in Python.
我知道一切都在呼唤
static PyObject *
gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
，结果从PyEval_EvalFrameEx返回，它看起来像是一个动态框架结构，我可以理解为stack还是什么?
and the result is returned from PyEval_EvalFrameEx which looks like it's a dynamic frame struct, could I understand it as stack or something?
好吧，看起来Python在内存中存储了一些上下文(对吗?).似乎每次使用yield时，它都会创建一个生成器，并将上下文存储在内存中，尽管不是所有的函数和var.
Ok, It looks like Python stores some context in memory (am I right?). It looks like every time we use yield it creates a generator, and stores the context in memory, although not all of the functions and vars.
我知道如果我要解析大循环或大数据，那么收益率是惊人的，它可以节省大量内存并使其变得简单.但是我的一些同事喜欢在任何地方使用收益，就像回报一样.阅读和理解代码并不容易，Python为大多数函数存储了上下文，这些函数可能再也不会被调用.这是不好的做法吗?
I know if I have big loop or big data to parse, yield is amazing, it saves a lot of memory and make it simple. But some of my workmates like to use yield everywhere, just like return. It's not easy to read and understand the code, and Python stores context for most of the function that may never be called again. Is it a bad practice?
所以，问题是:
-
PyEval_EvalFrameEx的工作方式. - 内存使用量.
- 在任何地方使用yield都是不好的习惯.
- How does
PyEval_EvalFrameExwork. - Memory use of yield.
- Is it bad practice for using yield everywhere.
我发现如果我有一个生成器，函数gen_send_ex将被调用两次，为什么?
And I found if I have a generator, function gen_send_ex will be called twice, why?
def test():
while 1:
yield 'test here'
test().next()
它将调用gen_send_ex两次，第一次是没有args，第二次是args，并得到结果.
It will call gen_send_ex twice, first time with no args, with args the second time, and get the result.
感谢您的耐心等候.
推荐答案
我看过这些文章:
本文告诉我PyEval_EvalFrameEx如何工作.
This article tell me how does PyEval_EvalFrameEx work.
http://tech .blog.aknin.name/2010/09/02/pythons-innards-hello-ceval-c-2/
本文告诉我Python的框架结构.
This article tell me the frame struct in Python.
http://tech.blog.aknin .name/2010/07/22/pythons-innards-interpreter-stacks/
这两个东西对我们非常重要.
These two stuff are very important for us.
所以让我自己回答我的问题.我不知道我是否正确.
如果我有误会或完全错误，请告诉我.
如果我有代码:
def gen():
count = 0
while count < 10:
count += 1
print 'call here'
yield count
那是一个非常简单的生成器.
That is a very simple generator.
f = gen()
每次调用它时，Python都会创建一个生成器对象.
And every time we call it, Python create a generator object.
PyObject *
PyGen_New(PyFrameObject *f)
{
PyGenObject *gen = PyObject_GC_New(PyGenObject, &PyGen_Type);
if (gen == NULL) {
Py_DECREF(f);
return NULL;
}
gen->gi_frame = f;
Py_INCREF(f->f_code);
gen->gi_code = (PyObject *)(f->f_code);
gen->gi_running = 0;
gen->gi_weakreflist = NULL;
_PyObject_GC_TRACK(gen);
return (PyObject *)gen;
}
我们可以看到它初始化了一个生成器对象.然后启动Frame.
We could see it init a generator object. And Init a Frame.
任何我们喜欢f.send()或f.next()的东西，它将调用gen_send_ex，以及下面的代码:
Anything we do like f.send() or f.next(), It will call gen_send_ex, and the code below:
static PyObject *
gen_iternext(PyGenObject *gen)
{
return gen_send_ex(gen, NULL, 0);
}
static PyObject *
gen_send(PyGenObject *gen, PyObject *arg)
{
return gen_send_ex(gen, arg, 0);
}
两个函数之间唯一的区别是arg，发送是发送arg，下一个发送NULL.
Only difference between two function is arg, send is send an arg, next send NULL.
下面的gen_send_ex代码:
gen_send_ex code below:
static PyObject *
gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
{
PyThreadState *tstate = PyThreadState_GET();
PyFrameObject *f = gen->gi_frame;
PyObject *result;
if (gen->gi_running) {
fprintf(stderr, "gi init\n");
PyErr_SetString(PyExc_ValueError,
"generator already executing");
return NULL;
}
if (f==NULL || f->f_stacktop == NULL) {
fprintf(stderr, "check stack\n");
/* Only set exception if called from send() */
if (arg && !exc)
PyErr_SetNone(PyExc_StopIteration);
return NULL;
}
if (f->f_lasti == -1) {
fprintf(stderr, "f->f_lasti\n");
if (arg && arg != Py_None) {
fprintf(stderr, "something here\n");
PyErr_SetString(PyExc_TypeError,
"can't send non-None value to a "
"just-started generator");
return NULL;
}
} else {
/* Push arg onto the frame's value stack */
fprintf(stderr, "frame\n");
if(arg) {
/* fprintf arg */
}
result = arg ? arg : Py_None;
Py_INCREF(result);
*(f->f_stacktop++) = result;
}
fprintf(stderr, "here\n");
/* Generators always return to their most recent caller, not
* necessarily their creator. */
Py_XINCREF(tstate->frame);
assert(f->f_back == NULL);
f->f_back = tstate->frame;
gen->gi_running = 1;
result = PyEval_EvalFrameEx(f, exc);
gen->gi_running = 0;
/* Don't keep the reference to f_back any longer than necessary. It
* may keep a chain of frames alive or it could create a reference
* cycle. */
assert(f->f_back == tstate->frame);
Py_CLEAR(f->f_back);
/* If the generator just returned (as opposed to yielding), signal
* that the generator is exhausted. */
if (result == Py_None && f->f_stacktop == NULL) {
fprintf(stderr, "here2\n");
Py_DECREF(result);
result = NULL;
/* Set exception if not called by gen_iternext() */
if (arg)
PyErr_SetNone(PyExc_StopIteration);
}
if (!result || f->f_stacktop == NULL) {
fprintf(stderr, "here3\n");
/* generator can't be rerun, so release the frame */
Py_DECREF(f);
gen->gi_frame = NULL;
}
fprintf(stderr, "return result\n");
return result;
}
好像Generator Object是它自己的Frame(称为gi_frame)的控制器.
Looks like Generator Object is a controller of it's own Frame which called gi_frame.
我添加了一些fprintf(...)，所以让我们运行代码.
I add some fprintf (...), so let's run code.
f.next()
f->f_lasti
here
call here
return result
1
因此，首先转到f_lasti(这是最后执行的指令的字节代码的整数偏移量，初始化为-1)，是-1，但没有args，然后功能继续.
So, first it goes to f_lasti(This is a integer offset into the byte code of the last instructions executed, initialized to -1), and yes it's -1, but with no args, then function goes on.
然后转到here，现在最重要的是PyEval_EvalFrameEx. PyEval_EvalFrameEx实现了CPython的评估循环，我们可以让它运行所有代码(实际上是Python操作码)，并运行print 'call here'行，它会打印文本.
Then goto here, the most important thing now is PyEval_EvalFrameEx. PyEval_EvalFrameEx implements CPython’s evaluation loop, we could thing it runs every code (in fact is Python opcode), and run the line print 'call here', it print text.
当代码转到yield时，Python使用框架对象存储上下文(我们可以搜索调用堆栈).放弃价值并放弃对代码的控制.
When code goes to yield, Python stores context by using frame object (we could search Call Stack). Give value back and give up control of code.
完成所有操作后，然后输入return result，并在终端中显示值1.
After everything done, then return result, and showing value 1 in terminal.
下次我们运行next()时，它将不会进入f_lasti范围.它显示:
Next time we run next(), it will not go to f_lasti scope. It shows:
frame
here
call here
return result
2
我们没有发送arg，因此仍然从PyEval_EvalFrameEx获取结果，结果为2.
We did not send arg so still get result from PyEval_EvalFrameEx and result is 2.
这篇关于yield如何在Python C代码中正常工作&不良部分的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
