交互式Python中的多处理程序包 [英] multiprocessing package in interactive Python

问题描述

我有以下代码test.py:

I have the following code test.py:

#multiprocessing in the interactive Python 

import time
from multiprocessing import Process, Pipe

def MyProcess(a):

    while(1):
       time.sleep(1)
       a.send("tic")    

if __name__ == "__main__":

    a, b = Pipe() 

    p = Process(target=MyProcess, args=(a,))
    p.start()

    while(1):
       msg=b.recv()
       print(msg)

如果我在DOS shell"python test.py"中执行它，它将很好地工作 但是，如果我使用IEP(Pyzo)的执行文件"，则无法使用.

It works fine if I execute it in the DOS shell "python test.py" But it doesn't work if I use "Execute File" from IEP (Pyzo).

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\pyzo2014a_64b\lib\multiprocessing\spawn.py", line 106, in spawn_main
    exitcode = _main(fd)
  File "C:\pyzo2014a_64b\lib\multiprocessing\spawn.py", line 116, in _main
    self = pickle.load(from_parent)
AttributeError: Can't get attribute 'MyProcess' on <module '__main__' (built-in)>

我发现这是一个已记录的问题".请检查下面链接的答案.

I found that this is a documented 'issue'. Please check the answer of the link below.

交互式模式下的多处理中断

这是否意味着我不应该使用交互式Python中的多处理程序包?这是否意味着我无法从IPython控制台创建进程? 对此的任何澄清将受到高度赞赏

Does it mean that I should not use multiprocessing package from the interactive Python? Does it mean I can not create a process from the IPython console? Any clarification on this will be highly appreciated

推荐答案

正确的，您不能使用解释器中的multiprocessing…主要是因为pickle不知道如何序列化交互功能.但是，如果使用名为pathos.multiprocessing的multiprocessing分支，则可以从解释器执行所需的操作.之所以起作用，是因为pathos.multiprocessing使用dill，并且dill知道如何序列化解释器中定义的函数(和其他对象).

Correct, you can't use multiprocessing from the interpreter… primarily because pickle doesn't know how to serialize interactive functions. However, if you use a multiprocessing fork, called pathos.multiprocessing, you can do what you want from the interpreter. This works because pathos.multiprocessing uses dill, and dill knows how to serialize functions (and other objects) that are defined in the interpreter.

>>> from pathos.multiprocessing import ProcessingPool as Pool
>>> 
>>> p = Pool(4)
>>> def squared(x):
...   return x**2
... 
>>> def pow(x,y):
...   return x**y
... 
>>> a = range(10)
>>> b = range(10,0,-1)
>>> 
>>> p.map(squared, a)
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> res = p.amap(pow, a, b)
>>> print "asynchronous, and with multiple inputs!"
asynchronous, and with multiple inputs!
>>> res.get()
[0, 1, 256, 2187, 4096, 3125, 1296, 343, 64, 9]

在此处获取pathos: https://github.com/uqfoundation

这篇关于交互式Python中的多处理程序包的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！