将sympy lambda传递给multiprocessing.Pool.map [英] Passing sympy lambda to multiprocessing.Pool.map

问题描述

我想并行执行一个sympy lambda函数. 我不知道:

I want to execute a sympy lambda function in parallel. I don't know:

• 为什么它虽然是lambda函数却可以并行工作
• 为什么我在没有池的情况下尝试执行时会停止工作
• 如果取消注释lambdify
中的第一个返回值，为什么会起作用

• why it works in parallel although it is a lambda function
• why it stops working when I try executing without the pool
• why it works if I uncomment the first return in lambdify

显然，降价预处理器在代码上方需要一行文本，因此这是代码:

And apparently the markdown preprocessor needs a line of text above the code so this is the code:

from multiprocessing import Pool

import sympy
from sympy.abc import x

def f(m):
    return m.lambdify()(1)

class Mult():
    def lambdify(self):
        # return sympy.lambdify(x, 2*x, 'numpy')
        self._lambdify = sympy.lambdify(x, 2 * x, 'numpy')
        return self._lambdify

if __name__ == '__main__':
    with Pool() as pool:
        m = Mult()
        print(pool.map(f, [m]))
        print(pool.map(f, [m]))
        print(f(m))
        print(pool.map(f, [m]))

它打印:

[2]
[2]
2
PicklingError: Can't pickle <function <lambda> at 0x000000000DF0D048>: attribute lookup <lambda> on numpy failed

(我削减了回溯)

如果我取消注释，它可以正常工作:

If I uncomment, it works normally:

[2]
[2]
2
[2]

我仅在Windows上进行了测试，它与"numexpr"(而不是"numpy")完全相同.

I tested only on Windows and it works exactly the same with 'numexpr' instead of 'numpy'.

推荐答案

对象Mult在创建时没有字段.因此，可以使用stock pickle库对其进行腌制.然后，当您调用lambdify时，会将_lambdify属性添加到包含lambda表达式的对象中，该表达式不能被腌制.这会导致map函数失败

The object Mult has no fields when it is created. It can thus be pickled with the stock pickle library. Then, when you call lambdify, you add a _lambdify attribute to the object containing a lambda expression, which cannot be pickled. This causes a failure in the map function

这说明了为什么在调用lambdify之前可以腌制对象并使用Pool.map以及为什么调用后它失败的原因. 取消注释lambdify中的行时，不会将属性添加到类中，并且在调用lambdify之后仍然可以腌制Mult对象.

This explains why before calling lambdify you can pickle the object and use Pool.map and why it fails after the call. When you uncomment the line in lambdify, you do not add the attribute to the class, and the Mult object can still be pickled after calling lambdify.

这篇关于将sympy lambda传递给multiprocessing.Pool.map的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！