具有多个参数的多处理功能可在Python 2.7中运行 [英] Multiprocessing with multiple arguments to function in Python 2.7

问题描述

我正在尝试实现多处理以加快复制循环，但是无法使其在Python27中工作.这是我程序的非常简化的版本，基于此处的文档和其他答案(例如

I'm trying to implement multiprocessing to speed up a replication loop, but cannot get it to work in Python27. This is a very simplified version of my program, based on the docs and other answers here at SO (e.g. Python multiprocessing pool.map for multiple arguments). I realize that there are a number of quesions on multiprocessing, but so far I haven't been able to solve this issue. Hopefully I haven't overlooked anything too trivial.

代码

import itertools
from multiprocessing import Pool

def func(g, h, i):
    return g + h + i

def helper(args):
    args2 = args[0] + (args[1],)
    return func(*args2)

pool = Pool(processes=4)
result = pool.map(helper, itertools.izip(itertools.repeat((2, 3)), range(20)))
print result

在使用map(...)时有效，但在使用pool.map(...)时无效.

This works when using map(...), but not when using pool.map(...).

错误消息:

Process PoolWorker-3:
Traceback (most recent call last):
File "C:\Program_\EPD_python27\lib\multiprocessing\process.py", line 258, in _
bootstrap
self.run()
File "C:\Program_\EPD_python27\lib\multiprocessing\process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "C:\Program_\EPD_python27\lib\multiprocessing\pool.py", line 85, in worker
task = get()
File "C:\Program_\EPD_python27\lib\multiprocessing\queues.py", line 376, in get
return recv()
AttributeError: 'module' object has no attribute 'helper'

推荐答案

通过添加main()函数如下来解决该问题:

The problem is solved by adding a main() function as:

import itertools
from multiprocessing import Pool

def func(g, h, i):
    return g + h + i

def helper(args):
    args2 = args[0] + (args[1],)
    return func(*args2)

def main():
    pool = Pool(processes=4)
    result = pool.map(helper,itertools.izip(itertools.repeat((2, 3)), range(10)))
    print result

if __name__ == '__main__':
    main()

基于@ErikAllik的回答，我认为这可能是Windows特有的问题.

Based on the answer from @ErikAllik I'm thinking that this might be a Windows-specific problem.

修改:这是一个清晰且内容丰富的关于多处理的教程在python中.

edit: Here is a clear and informative tutorial on multiprocessing in python.

这篇关于具有多个参数的多处理功能可在Python 2.7中运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！