Lambdify可与Python一起使用，但Cython会引发异常 [英] Lambdify works with Python, but throws an exception with Cython

问题描述

我的网站运行此Python脚本，如果使用Cython，该脚本将得到更优化.最近，我需要添加使用Lambdify的Sympy ，但效果不佳与Cython.

My website runs this Python script that would be way more optimized if Cython is used. Recently I needed to add Sympy with Lambdify, and this is not going well with Cython.

因此，我将问题简化为一个最小的工作示例.在代码中，我有一个字典，其中包含具有列表值的字符串键.我想将这些键用作变量.在下面的简化示例中，只有1个变量，但是通常我需要更多.请检查以下示例:

So I stripped the problem to a minimum working example. In the code, I have a dictionary with string keys with values that are lists. I would like to use these keys as variables. In the following simplified example, there's only 1 variable, but generally I need more. Please check the following example:

import numpy as np
from sympy.parsing.sympy_parser import parse_expr
from sympy.utilities.lambdify import lambdify, implemented_function
from sympy import S, Symbol
from sympy.utilities.autowrap import ufuncify


def CreateMagneticFieldsList(dataToSave,equationString,DSList):

    expression  = S(equationString)
    numOfElements = len(dataToSave["MagneticFields"])

    #initialize the magnetic field output array
    magFieldsArray    = np.empty(numOfElements)
    magFieldsArray[:] = np.NaN

    lam_f = lambdify(tuple(DSList),expression,modules='numpy')
    try:
        # pass
        for i in range(numOfElements):
            replacementList = np.zeros(len(DSList))


            for j in range(len(DSList)):
                replacementList[j] = dataToSave[DSList[j]][i]

            try:
                val = np.double(lam_f(*replacementList))

            except:
                val = np.nan
            magFieldsArray[i] = val
    except:
        print("Error while evaluating the magnetic field expression")
    return magFieldsArray


list={"MagneticFields":[1,2,3,4,5]}

out=CreateMagneticFieldsList(list,"MagneticFields*5.1",["MagneticFields"])

print(out)

我们称之为test.py.这很好.现在，我想对此进行cythonize，所以我使用以下脚本:

Let's call this test.py. This works very well. Now I would like to cythonize this, so I use the following script:

#!/bin/bash

cython --embed -o test.c test.py
gcc -pthread -fPIC -fwrapv -Ofast -Wall -L/lib/x86_64-linux-gnu/ -lpython3.4m -I/usr/include/python3.4 -o test.exe test.c

现在，如果我执行./test.exe，它将引发异常！例外:

Now if I execute ./test.exe, it throws an exception! Here's the exception:

Traceback (most recent call last):
  File "test.py", line 42, in init test (test.c:1811)
    out=CreateMagneticFieldsList(list,"MagneticFields*5.1",["MagneticFields"])
  File "test.py", line 19, in test.CreateMagneticFieldsList (test.c:1036)
    lam_f = lambdify(tuple(DSList),expression,modules='numpy')
  File "/usr/local/lib/python3.4/dist-packages/sympy/utilities/lambdify.py", line 363, in lambdify
    callers_local_vars = inspect.currentframe().f_back.f_locals.items()
AttributeError: 'NoneType' object has no attribute 'f_locals'

所以问题是:我怎么能lambdify与Cython一起使用?

So the question is: How can I get lambdify to work with Cython?

注意事项:我想指出我有Debian Jessie，这就是为什么我使用Python 3.4.我还要指出，当不使用lambdify时，我对Cython没任何问题.我还要指出，Cython已使用pip3 install cython --upgrade更新到最新版本.

Notes: I would like to point out that I have Debian Jessie, and that's why I'm using Python 3.4. Also I would like to point out that I don't have any problem with Cython when not using lambdify. Also I would like to point out that Cython is updated to the last version with pip3 install cython --upgrade.

推荐答案

这是针对实际问题(在注释和@jjhakala的答案中已确定)的解决方案，因为Cython不会为该问题生成完整的回溯/内省信息编译函数.我从您的评论中收集到，出于速度原因，您希望使用Cython编译大多数程序.

This is a something of a workround to the real problem (identified in the comments and @jjhakala's answer) that Cython doesn't generate full tracebacks/introspection information for compiled functions. I gather from your comments that you'd like to keep most of your program compiled with Cython for speed reasons.

解决方案"是仅将Python解释器用于需要调用lambdify并将其余部分保留在Cython中的单个函数.您可以使用exec.

The "solution" is to use the Python interpreter for only the individual function that needs to call lambdify and leave the rest in Cython. You can do this using exec.

这个想法的一个非常简单的例子是

A very simple example of the idea is

exec("""
def f(func_to_call):
    return func_to_call()""")

# a Cython compiled version    
def f2(func_to_call):
    return func_to_call())

这可以编译为Cython模块并导入，导入后，Python解释器将在字符串中运行代码，并将f正确添加到模块全局变量中.如果我们创建一个纯Python函数

This can be compiled as a Cython module and imported, and upon being imported the Python interpreter runs the code in the string, and correctly adds f to the module globals. If we create a pure Python function

def g():
    return inspect.currentframe().f_back.f_locals

调用cython_module.f(g)给我一个带有键func_to_call的字典(如预期)，而cython_module.f2(g)给我一个__main__模块全局变量(但这是因为我从解释器运行而不是使用--embed ).

calling cython_module.f(g) gives me a dictionary with key func_to_call (as expected) while cython_module.f2(g) gives me the __main__ module globals (but this is because I'm running from an interpreter rather than using --embed).

编辑:基于您的代码的完整示例

Full example, based on your code

from sympy import S, lambdify # I'm assuming "S" comes from sympy
import numpy as np

CreateMagneticFieldsList = None # stops a compile error about CreateMagneticFieldsList being undefined

exec("""def CreateMagneticFieldsList(dataToSave,equationString,DSList):

    expression  = S(equationString)
    numOfElements = len(dataToSave["MagneticFields"])

    #initialize the magnetic field output array
    magFieldsArray    = np.empty(numOfElements)
    magFieldsArray[:] = np.NaN

    lam_f = lambdify(tuple(DSList),expression,modules='numpy')
    try:
        # pass
        for i in range(numOfElements):
            replacementList = np.zeros(len(DSList))


            for j in range(len(DSList)):
                replacementList[j] = dataToSave[DSList[j]][i]

            try:
                val = np.double(lam_f(*replacementList))

            except:
                val = np.nan
            magFieldsArray[i] = val
    except:
        print("Error while evaluating the magnetic field expression")
    return magFieldsArray""")


list={"MagneticFields":[1,2,3,4,5]}

out=CreateMagneticFieldsList(list,"MagneticFields*5.1",["MagneticFields"])
print(out)

使用您的脚本编译时，会打印

When compiled with your script this prints

[5.1 10.2 15.3 20.4 25.5]

[ 5.1 10.2 15.3 20.4 25.5 ]

基本上，我所做的只是将您的函数包装在exec语句中，因此它由Python解释器执行.这部分不会从Cython中获得任何好处，但是您程序的其余部分仍然会受益.如果要最大化使用Cython编译的数量，可以将其划分为多个函数，以便exec中只有一小部分包含lambdify.

Substantially all I've done is wrapped your function in an exec statement, so it's executed by the Python interpreter. This part won't see any benefit from Cython, however the rest of your program still will. If you want to maximise the amount compiled with Cython you could divide it up into multiple functions so that only the small part containing lambdify is in the exec.

这篇关于Lambdify可与Python一起使用，但Cython会引发异常的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！