itertools中的izip_longest：迭代器中的indexError如何工作？ [英] izip_longest in itertools: How does rasing IndexError inside the iterator work?
问题描述
在这个问题中@lazyr询问以下内容来自此处<的代码 izip_longest 迭代器/ a>有效：
In this question @lazyr asks how the following code of izip_longest iterator from here works:
def izip_longest_from_docs(*args, **kwds):
# izip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
fillvalue = kwds.get('fillvalue')
def sentinel(counter = ([fillvalue]*(len(args)-1)).pop):
yield counter() # yields the fillvalue, or raises IndexError
fillers = repeat(fillvalue)
iters = [chain(it, sentinel(), fillers) for it in args]
try:
for tup in izip(*iters):
yield tup
except IndexError:
pass
当我试图理解它是如何工作的时候，我偶然发现了一个问题：
如果 IndexError 在里面引发其中一个迭代器被发送到 izip_longest 作为参数？。
When I was trying to understand how it works I stumbled into the question:
"What if IndexError is raised inside one of those iterators that are sent to izip_longest as parameters?".
然后我写了一些测试代码：
Then I wrote some testing code:
from itertools import izip_longest, repeat, chain, izip
def izip_longest_from_docs(*args, **kwds):
# The code is exactly the same as shown above
....
def gen1():
for i in range(5):
yield i
def gen2():
for i in range(10):
if i==8:
raise IndexError #simulation IndexError raised inside the iterator
yield i
for i in izip_longest_from_docs(gen1(),gen2(), fillvalue = '-'):
print('{i[0]} {i[1]}'.format(**locals()))
print('\n')
for i in izip_longest(gen1(),gen2(), fillvalue = '-'):
print('{i[0]} {i[1]}'.format(**locals()))
结果证明中的函数是itertools 模块和 izip_longest_from_docs 工作方式不同。
And it turned out that the function in itertools module and izip_longest_from_docs work differently.
上述代码的输出：
>>>
0 0
1 1
2 2
3 3
4 4
- 5
- 6
- 7
0 0
1 1
2 2
3 3
4 4
- 5
- 6
- 7
Traceback (most recent call last):
File "C:/..., line 31, in <module>
for i in izip_longest(gen1(),gen2(), fillvalue = '-'):
File "C:/... test_IndexError_inside iterator.py", line 23, in gen2
raise IndexError
IndexError
因此，可以清楚地看到，的代码izip_longes 来自 itertools 确实传播了 IndexError 异常（我认为应该如此），但 izip_longes_from_docs '吞噬' IndexError 异常，因为它将来自 sentinel 的信号停止迭代。
So, it's clearly seen, that the code of izip_longes from itertools did propagate IndexError exception (as I think it should), but izip_longes_from_docs 'swallowed' IndexError exception as it took it as a signal from sentinel to stop iterating.
我的问题是，他们是如何在
My question is, how did they worked around IndexError propagation in the code in theitertools module?
推荐答案
in <$ c 代码中的$ c> izip_longest_next izip_longest ，没有使用过哨兵。
in izip_longest_next in the code of izip_longest, no sentinel is used.
相反，CPython会跟踪有多少迭代器仍然在计数器处于活动状态，并在活动数量达到零时停止。
Instead, CPython keeps track of how many of the iterators are still active with a counter, and stops when the number active reaches zero.
如果发生错误，它将结束迭代，就像没有迭代器仍处于活动状态一样，并允许要传播的错误。
If an error occurs, it ends iteration as if there are no iterators still active, and allows the error to propagate.
代码：
item = PyIter_Next(it);
if (item == NULL) {
lz->numactive -= 1;
if (lz->numactive == 0 || PyErr_Occurred()) {
lz->numactive = 0;
Py_DECREF(result);
return NULL;
} else {
Py_INCREF(lz->fillvalue);
item = lz->fillvalue;
PyTuple_SET_ITEM(lz->ittuple, i, NULL);
Py_DECREF(it);
}
}
我看到的最简单的解决方案：
The simplest solution I see:
def izip_longest_modified(*args, **kwds):
# izip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
fillvalue = kwds.get('fillvalue')
class LongestExhausted(Exception):
pass
def sentinel(counter = ([fillvalue]*(len(args)-1)).pop):
try:
yield counter() # yields the fillvalue, or raises IndexError
except:
raise LongestExhausted
fillers = repeat(fillvalue)
iters = [chain(it, sentinel(), fillers) for it in args]
try:
for tup in izip(*iters):
yield tup
except LongestExhausted:
pass
这篇关于itertools中的izip_longest：迭代器中的indexError如何工作？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
