Python多重处理:AttributeError:'Test'对象没有属性'get_type' [英] Python Multiprocessing: AttributeError: 'Test' object has no attribute 'get_type'
问题描述
简短的简短版本:
我无法并行化使用实例方法的代码.
I am having trouble parallelizing code which uses instance methods.
长版:
此python代码产生错误:
This python code produces the error:
Error
Traceback (most recent call last):
File "/Users/gilzellner/dev/git/3.2.1-build/cloudify-system-tests/cosmo_tester/test_suites/stress_test_openstack/test_file.py", line 24, in test
self.pool.map(self.f, [self, url])
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/pathos/multiprocessing.py", line 131, in map
return _pool.map(star(f), zip(*args)) # chunksize
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/multiprocess/pool.py", line 251, in map
return self.map_async(func, iterable, chunksize).get()
File "/Users/gilzellner/.virtualenvs/3.2.1-build/lib/python2.7/site-packages/multiprocess/pool.py", line 567, in get
raise self._value
AttributeError: 'Test' object has no attribute 'get_type'
这是我遇到的一个实际问题的简化版本.
This is a simplified version of a real problem I have.
import urllib2
from time import sleep
from os import getpid
import unittest
from pathos.multiprocessing import ProcessingPool as Pool
class Test(unittest.TestCase):
def f(self, x):
print urllib2.urlopen(x).read()
print getpid()
return
def g(self, y, z):
print y
print z
return
def test(self):
url = "http://nba.com"
self.pool = Pool(processes=1)
for x in range(0, 3):
self.pool.map(self.f, [self, url])
self.pool.map(self.g, [self, url, 1])
sleep(10)
由于这里的建议,我正在使用pathos.multiprocessing: 多处理:池和泡菜错误-酸洗错误:不能腌制< type'instancemethod'> ;:属性查找__builtin __.instancemethod失败
I am using pathos.multiprocessing due to the recommendation here: Multiprocessing: Pool and pickle Error -- Pickling Error: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed
在使用pathos.multiprocessing之前,错误是:
Before using pathos.multiprocessing, the error was:
"PicklingError: Can't pickle <type 'instancemethod'>: attribute lookup __builtin__.instancemethod failed"
推荐答案
您不正确地使用了多处理map
方法.
根据 python文档:>
You're using multiprocessing map
method incorrectly.
According to python docs:
与map()内置函数的并行等效项(仅支持 一个可迭代的论点).
A parallel equivalent of the map() built-in function (it supports only one iterable argument though).
标准map
:
将函数应用于所有iterable并返回列表 结果.
Apply function to every item of iterable and return a list of the results.
用法示例:
from multiprocessing import Pool
def f(x):
return x*x
if __name__ == '__main__':
p = Pool(5)
print(p.map(f, [1, 2, 3]))
您正在寻找的是 apply_async 方法:
def test(self):
url = "http://nba.com"
self.pool = Pool(processes=1)
for x in range(0, 3):
self.pool.apply_async(self.f, args=(self, url))
self.pool.apply_async(self.g, args=(self, url, 1))
sleep(10)
这篇关于Python多重处理:AttributeError:'Test'对象没有属性'get_type'的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!