大型迭代器(itertools)的笛卡尔积 [英] Cartesian product of large iterators (itertools)

问题描述

从一个上一个问题中我学到了一些有趣的东西.如果Python的itertools.product被提供了一系列迭代器，则这些迭代器将在笛卡尔积开始之前的 中转换为元组. 相关 查看itertools.product的源代码得出的结论是，尽管没有中间结果存储在内存中，但是元组版本原始迭代器是在产品迭代开始之前创建的.

From a previous question I learned something interesting. If Python's itertools.product is fed a series of iterators, these iterators will be converted into tuples before the Cartesian product begins. Related questions look at the source code of itertools.product to conclude that, while no intermediate results are stored in memory, tuple versions of the original iterators are created before the product iteration begins.

问题:当(经过元组转换的)输入太大而无法保存在内存中时，是否有一种方法可以创建笛卡尔乘积的迭代器?简单的例子:

Question: Is there a way to create an iterator to a Cartesian product when the (tuple converted) inputs are too large to hold in memory? Trivial example:

import itertools
A = itertools.permutations(xrange(100))
itertools.product(A)

一个更实际的用例将采用一系列(*iterables[, repeat])，就像该函数的原始实现一样-上面只是一个示例.看来您无法使用itertools.product的当前实现，所以我欢迎使用纯python提交(尽管您不能击败itertools的C后端！).

A more practical use case would take in a series of (*iterables[, repeat]) like the original implementation of the function - the above is just an example. It doesn't look like you can use the current implementation of itertools.product, so I welcome in submission in pure python (though you can't beat the C backend of itertools!).

推荐答案

下面是一个实现，该实现调用可调用对象并迭代可迭代对象(假定可重新启动):

Here's an implementation which calls callables and iterates iterables, which are assumed restartable:

def product(*iterables, **kwargs):
    if len(iterables) == 0:
        yield ()
    else:
        iterables = iterables * kwargs.get('repeat', 1)
        it = iterables[0]
        for item in it() if callable(it) else iter(it):
            for items in product(*iterables[1:]):
                yield (item, ) + items

测试:

import itertools
g = product(lambda: itertools.permutations(xrange(100)),
            lambda: itertools.permutations(xrange(100)))
print next(g)
print sum(1 for _ in g)

这篇关于大型迭代器(itertools)的笛卡尔积的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！