悲哀的多重处理不能泡菜 [英] pathos multiprocessing cannot pickle

问题描述

我遇到与此 person 相似的问题.我无法在pathos模块中运行简单的多处理例程，并收到一个酸洗错误.下面是代码和错误.

I am having a similar issue to this person. I am unable to run a simple multiprocessing routine in the pathos module and receive a pickling error. Below is the code and error.

from pathos.multiprocessing import ProcessingPool
import dill
class ProcClass(object):
    def __init__(self):
        pass
    def f(self,x):
        return x*x
pc = ProcClass()
pl = ProcessingPool(3)
print pl.map(pc.f, range(10))

返回的错误:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/threading.py", line 551, in __bootstrap_inner
    self.run()
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/threading.py", line 504, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/multiprocessing/pool.py", line 320, in _handle_tasks
    put(task)
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

我已经安装:pp，dill，pox和多处理，然后安装了pathos.安装有效，但始终会出现此错误:

I have installed: pp, dill, pox and multiprocessing then installed pathos. The install works but always gives me this error:

警告:未解决以下依赖关系之一: pp(ft)> = 1.6.4.5 莳萝> = 0.2.4 痘> = 0.2.2 (多)处理

WARNING: One of the following dependencies is unresolved: pp(ft) >=1.6.4.5 dill >=0.2.4 pox >=0.2.2 (multi)processing

根据pathos的作者对类似问题的回答，看来安装存在问题.我已经删除并重新安装了几次，每次都验证是否安装了正确的依赖关系.我在MacOS上运行并使用python 2.7. 任何帮助将不胜感激！

Based on a response from the author of pathos to a similar question, it looks like there is a problem with the install. I have removed and reinstalled several times, each time verifying the proper dependancies are installed. I'm running on MacOS and using python 2.7. Any help will be greatly appreciated!

推荐答案

似乎您缺少关键的依赖项.对于pathos，您应该使用multiprocess，它是multiprocessing的一个分支，它使用dill序列化器.

It appears that you are missing a critical dependency. With pathos, you should use multiprocess, which is a fork of multiprocessing which uses the dill serializer.

>>> import multiprocessing
>>> multiprocessing.__version__
'0.70a1'
>>> import multiprocess   
>>> multiprocess.__version__
'0.70.4.dev0'
>>> 
>>> multiprocess.Pool().map(lambda x:x*x, range(10))
[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
>>> 

这篇关于悲哀的多重处理不能泡菜的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！