如何组成迭代器? [英] How to compose iterators?

问题描述

我有一个在其间传递结构化数据的节点网络.对于我的子问题，我们有一个分支-节点的线性序列:

I have a network of nodes passing structured data in between. For my subproblem, we have this branch - linear sequence of nodes:

nodes = [source, n1, n2, n3, n4]

第一个节点是生成器，每个其他节点从输入节点获取值并提供输出值.当前的实现是从管道到put()到管道的普通get()，并且每个节点都有单独的线程(这是有原因的).我想将其更改为yield -ing迭代器.

First node is a generator, each other node gets values from input node and provides output value. Current implementation is plain get() from a pipe and put() to a pipe and there is separate thread per node (there is reason for that). I want to change it to a yield-ing iterator.

我想以以下方式进行评估(如果我们认为节点是可调用的):

And I want to evaluate in a following way (if we consider node to be a callable):

for result in n4(n3(n2(n1(source()))):
    print result

我想象评估上下文的构建是这样的:

I imagine the construction of evaluation context like this:

context = src
for node in nodes[1:]:
    context = pipe(context, node)

for result in context:
    print result

限制:

我仍然希望能够单独使用节点-而不是嵌套，通过其他方式传递数据，因为节点可能位于单独的线程中.示例:[source, n1,n2]在一个线程中(可能是嵌套的)，[n3, n4]在另一个线程中(可能是嵌套的)，数据在n2和n3之间进行管道传输.情况:可能有一个非线性节点图，我想以此方式对分支进行分组.

I still want to be able to use nodes separately - not nested, piping data by other means, because the nodes might be in separate threads. Example: [source, n1,n2] in one thread (might be nested), [n3, n4] in the other (might be nested), data piped between n2 and n3. Case: there might be a non-linear node graph where I want to group branches this way.

node必须是一个可保持计算状态的类

node has to be a class to hold computation state

context和pipe(context, node)的实现看起来如何?或者，如果可以通过其他方式解决它，您有任何提示吗?

How the implementation of the context and the pipe(context, node) might look like? Or if it can be solved in a different way, do you have any hints?

Python 3.3中的yield from是否可以(PEP380)帮助我解决问题以任何方式?

Can yield from in Python 3.3 (PEP380) help my case in any way?

推荐答案

如果要编写任意数量的函数(或可调用函数)，请使用

If all you want is to compose arbitrary numbers of functions (or callables), use the compose_mult recipe from the functional module documentation.

使用该解决方案的解决方案:

A solution which uses that:

from functional import compose, foldr, partial
from itertools  import imap
compose_mult = partial(reduce, compose) 
chain_nodes = lambda nodes: imap(compose_mult(nodes[1:]), nodes[0])
chain_gen_nodes = lambda nodes: imap(compose_mult((g.send for g in nodes[1:])), nodes[0])


# equivalent not as a one-liner
#def chain_nodes(nodes):
#    source = nodes[0]
#    composed_nodes = compose_mult(nodes[1:])
#    return (composed_nodes(x) for x in source)

如果节点是接受输入的生成器(通过send)，则使用chain_gen_nodes提取其发送功能.

If the nodes are generators that accept input (via send), then use chain_gen_nodes, which extracts their send function.

但是，请注意，不允许将send连接到刚启动的生成器(因为必须在yield的位置才能接收该值).这是您必须要处理的事情，例如，在生成器yield的第一个迭代中使其成为虚拟值，并在将它们发送到chain_nodes之前的某个时候进行处理.或者，您可以仅将节点保留为普通可调用对象.

Note, however, that one is not allowed to send to a just-started generator (because it has to be at the point of a yield to receive the value). This is something you are going to have to handle yourself, such as by having your generators yield a dummy value on their first iteration, and advancing them at some point before sending them to chain_nodes. Or you could just keep your nodes as ordinary callables.

如果您确实需要将迭代器前进一步，请执行以下操作:next(izip(*nodes[1:]))

If you do need to advance the iterators one step: next(izip(*nodes[1:]))

这篇关于如何组成迭代器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！