functools.partial 不能与 multiprocessing.Pool.map 一起使用吗? [英] Does functools.partial not work with multiprocessing.Pool.map?

问题描述

我的代码简化后如下所示:

I have code that, simplified down, looks like this:

run = functools.partial(run, grep=options.grep, print_only=options.print_only, force=options.force)

if not options.single and not options.print_only and options.n > 0:
    pool = multiprocessing.Pool(options.n)
    Map = pool.map
else: Map = map

for f in args:
    with open(f) as fh: Map(run, fh)

try:
    pool.close()
    pool.join()
except NameError: pass

当我在单进程模式下运行它时效果很好，但由于这样的错误而失败

That works fine when I run it in single process mode, but fails with errors like this

TypeError: type 'partial' takes at least one argument

通过多处理模块与长调用堆栈混合在一起.怎么回事?

mixed up together with long call stacks through the multiprocessing module. What's going on?

我使用的是 python 2.6.1.

I'm using python 2.6.1.

推荐答案

Google 告诉我这是一个 错误蟒蛇;显然已在 Py3k 中修复.据说是因为 partial 不可pickle.

Google tells me that this is a bug in Python; apparently fixed in Py3k. It's supposedly due to partial not being picklable.

有一个解决方法.

这篇关于functools.partial 不能与 multiprocessing.Pool.map 一起使用吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！