ipython并行中的异步评估 [英] Asynchronous evaluation in ipython parallel

问题描述

由于很棒的 1.0.0 版本，我一直在玩 iPython 并行接口。我要做的是建立一个异步随机梯度下降系统。我看到它的方式，我想向所有节点发送一个函数，并在结果出来时得到结果。从我能够实现的内容和文档中瞥一眼，实现的标准视图并不真正支持这一点。 get（timeout）方法会这样做，但你无法真正遍历< ASync_result> 使用超时的对象。我设法让它运行的方式是以下

Since the awesome 1.0.0 release I've been playing around with iPython parallel interface. What I'm trying to do is to set up a asynchronous stochastic gradient descent system. The way I see it, I want to send a function to all the nodes and get the results as they come out. From what I was able to implement and glance from the documentation the standard views implemented don't really support that. The get(timeout) method would do that, but you can't really loop through every entry in a <ASync_result> object using a timeout. The way I managed to get it running was the following

c = Client()
calls = []
for i,j in enumerate(args):
    calls.append( c[ i % len( c.ids ) ].apply( f, j ) )

while condition:
    dels = []
    for i,j in enumerate( calls ):
         try:
             print j.get(0.01) #or some other timeout
             dels.append( i ) #I keep track of the calls that have been called
             #do something with the last result, throw a new call
             calls.append( c[ i % len(c.ids) ].apply( f, argument )
         except:
             pass

    for i,d in enumerate( dels ):
         del calls[ d - i ] #delete gotten calls

    #evaluate stopping condition

现在，在你们大声尖叫之前，这是一个可怕的代码和一个愚蠢的方法来做到这一点，我知道。我可以做出这种特殊方式做得更好，但我是只是想知道是否有一些内置的方法在IPython.parallel中做类似的事情。

Now, before you all go screaming that this is horrible code and a stupid way to do that, I know it. I could make this particular way of doing it nicer, but I'm just wondering if there is some built-in way of doing something similar in IPython.parallel.

提前感谢任何花时间的人。

Thanks in advance to anyone taking the time.

最好，
Al。

Best, Al.

推荐答案

你可以创建多个异步调用，然后迭代它们。

You can create multiple async calls, and then iterate through them.

c = Client()
dview = c[:]
asyncs = [dview.map_async(f, [arg]) for arg in args]
while asyncs:
    for async in asyncs[:]:
        if async.ready():
            asyncs.remove(async)
            print async.result[0]

这篇关于ipython并行中的异步评估的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！