Python名称空间与ipython并行问题 [英] Python name space issues with ipython parallel

问题描述

我开始尝试使用IPython并行工具并遇到问题。我启动我的python引擎：

I'm starting to experiment with the IPython parallel tools and have an issue. I start up my python engines with:

ipcluster start -n 3

然后以下代码运行正常：

Then the following code runs fine:

from IPython.parallel import Client

def dop(x):
    rc = Client()
    dview = rc[:]
    dview.block=True
    dview.execute('a = 5')
    dview['b'] = 10
    ack = dview.apply(lambda x: a+b+x, x)
    return ack

ack = dop(27)
print ack

返回[42 ，42,42]应该如此。但是，如果我将代码分成不同的文件：
dop.py：

returns [42, 42, 42] as it should. But if I break the code into different files: dop.py:

from IPython.parallel import Client

def dop(x):
    rc = Client()
    dview = rc[:]
    dview.block=True
    dview.execute('a = 5')
    dview['b'] = 10
    print dview['a']
    ack = dview.apply(lambda x: a+b+x, x)
    return ack

并尝试以下操作：

from dop import dop
ack = dop(27)
print ack

我从每个引擎收到错误：

I get errors from each engine:

[0:apply]: NameError: global name 'a' is not defined
[1:apply]: NameError: global name 'a' is not defined
[2:apply]: NameError: global name 'a' is not defined

我不明白...为什么我不能把这个函数放在另一个文件中并导入它？

I don't get it...why can't I put the function in a different file and import it?

推荐答案

快速回答：装饰你的功能w如果您希望它能够访问引擎的全局命名空间，那么 @interactive 来自 IPython.parallel.util [1] ：

Quick answer: decorate your function with @interactive from IPython.parallel.util[1] if you want it to have access to the engine's global namespace:

from IPython.parallel.util import interactive
f = interactive(lambda x: a+b+x)
ack = dview.apply(f, x)

实际解释：

IPython用户命名空间本质上是模块 __ main __ 。当你执行执行（'a = 5'）时运行代码。

the IPython user namespace is essentially the module __main__. This is where code is run when you do execute('a = 5').

如果你定义一个交互式功能，其模块也是 __ main __ ：

If you define a function interactively, its module is also __main__:

lam = lambda x: a+b+x
lam.__module__
'__main__'

当引擎反序列化一个函数时，它会这样做在函数模块的相应全局命名空间中，客户端中 __ main __ 中定义的函数也在 __ main __ 中定义引擎，因此可以访问 a 。

When the Engine unserializes a function, it does so in the appropriate global namespace for the function's module, so functions defined in __main__ in your client are also defined in __main__ on the Engine, and thus have access to a.

将文件放入文件并导入后，函数不再附加到 __ main __ ，但模块 dop ：

Once you put it in a file and import it, then the functions are no longer attached to __main__, but the module dop:

from dop import dop
dop.__module__
'dop'

该模块中常规定义的所有函数（包括lambdas）都将具有此值，因此当它们在Engine上解包时，它们的全局命名空间将为 dop 模块的t，不 __ main __ ，因此您的'a'无法访问。

All functions conventionally defined in that module (lambdas included) will have this value, so when they are unpacked on the Engine their global namespace will be that of the dop module, not __main__, so your 'a' is not accessible.

出于这个原因，IPython提供了一个简单的 @interactive 装饰器，导致任何函数被解压缩，就像它一样无论函数在何处实际定义，都在 __ main __ 中定义。

For this reason, IPython provides a simple @interactive decorator that results in any function being unpacked as if it were defined in __main__, regardless of where the function is actually defined.

有关差异的示例，请参加这个 dop.py ：

For an example of the difference, take this dop.py:

from IPython.parallel import Client
from IPython.parallel.util import interactive

a = 1

def dop(x):
    rc = Client()
    dview = rc[:]
    dview['a'] = 5
    f = lambda x: a+x
    return dview.apply_sync(f, x)

def idop(x):
    rc = Client()
    dview = rc[:]
    dview['a'] = 5
    f = interactive(lambda x: a+x)
    return dview.apply_sync(f, x)

现在， dop 将使用' a'来自dop模块， i dop 将使用引擎命名空间中的a。两者之间的唯一区别是传递给apply的函数包含在 <$中c $ c> @interactive ：

Now, dop will use 'a' from the dop module, and idop will use 'a' from your engine namespaces. The only difference between the two is that the function passed to apply is wrapped in @interactive:

from dop import dop, idop
print dop(5)  # 6
print idop(5) # 10

[1]：在IPython中> = 0.13（即将推出的版本）， @interactive 也可以从IPython.parallel导入交互式获得，它始终应该是。

[1]: In IPython >= 0.13 (upcoming release), @interactive is also available as from IPython.parallel import interactive, where it always should have been.

这篇关于Python名称空间与ipython并行问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！