Python多处理库错误(AttributeError:__exit__) [英] Python Multiprocessing Lib Error (AttributeError: __exit__)

问题描述

使用pool.map(funct, iterable)时出现此错误:

AttributeError: __exit__

没有说明，仅堆栈跟踪到模块内的pool.py文件.

No Explanation, only stack trace to the pool.py file within the module.

以这种方式使用:

with Pool(processes=2) as pool:
   pool.map(myFunction, mylist)
   pool.map(myfunction2, mylist2)

我怀疑可挑剔性可能存在问题(python需要pickle，或将列表数据转换为字节流)，但我不确定这是否正确或是否要调试.

I suspect there could be a problem with the picklability (python needs to pickle, or transform list data into byte stream) yet I'm not sure if this is true or if it is how to debug.

产生此错误的新代码格式:

new format of code that produces this error :

def governingFunct(list):
    #some tasks
    def myFunction():
         # function contents
    with closing(Pool(processes=2)) as pool:
         pool.map(myFunction, sublist)
         pool.map(myFunction2, sublist2)

发生错误:

PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed

推荐答案

在Python 2.x和3.0、3.1和3.2中，multiprocessing.Pool()对象不是上下文管理器.您不能在with语句中使用它们.只有在Python 3.3及更高版本中，您才可以使用它们.从 Python 3 multiprocessing.Pool()文档:

In Python 2.x and 3.0, 3.1 and 3.2, multiprocessing.Pool() objects are not context managers. You cannot use them in a with statement. Only in Python 3.3 and up can you use them as such. From the Python 3 multiprocessing.Pool() documentation:

3.3版中的新功能:池对象现在支持上下文管理协议-请参阅上下文管理器类型. __enter__()返回池对象，并且__exit__()调用Terminate().

New in version 3.3: Pool objects now support the context management protocol – see Context Manager Types. __enter__() returns the pool object, and __exit__() calls terminate().

对于早期的Python版本，您可以使用 contextlib.closing() ，但要考虑到该帐户将称为 pool.close() ，而不是pool.terminate().在这种情况下，请手动终止:

For earlier Python versions, you could use contextlib.closing(), but take into account this'll call pool.close(), not pool.terminate(). Terminate manually in that case:

from contextlib import closing

with closing(Pool(processes=2)) as pool:
    pool.map(myFunction, mylist)
    pool.map(myfunction2, mylist2)
    pool.terminate()

或创建自己的terminating()上下文管理器:

or create your own terminating() context manager:

from contextlib import contextmanager

@contextmanager
def terminating(thing):
    try:
        yield thing
    finally:
        thing.terminate()

with terminating(Pool(processes=2)) as pool:
    pool.map(myFunction, mylist)
    pool.map(myfunction2, mylist2)

这篇关于Python多处理库错误(AttributeError:__exit__)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！