Cython具有“头等功能对象”，其功能包括： -他们的效率如何？他们有Python开销吗？ [英] Cython has "first class function objects" - how efficient are they? Do they have Python overhead?

问题描述

我有一个外部库，用于计算给定函数的最优值，例如最小值。说它的标头为我提供了一个函数

I have an external library that computes the optima, say minima, of a given function. Say its headers give me a function

double[] minimizer(ObjFun f)

标头定义的地方

typedef double (*ObjFun)(double x[])

我已经为此库生成了Cython包装器。我现在想给用户提供参数化的函数，具体地说，我想编写一个函数

I have generated Cython wrappers for this library. I now want to give user parameterized functions to it, specifically, I want to write a function

def getFunction(double q11, double q12, double q22):
    cdef f(double x[]):
        return x[0]*x[0]*q11 + 2*x[0]*x[1]*q12 +  x[1]*x[1]*q22
return f

带有其参数（q11，q12，q22）以获取可用作C优化库的回调的函数。

that the user calls with their parameters (q11, q12, q22) to get a function that can be used as a callback for the C optimization library.

（上面的示例是经过精心设计和简化的，在Cython中执行此操作的全部目的是希望将几乎C语言有效的回调传递给该库。）

(The example above is contrived and simplified, the whole point of doing this in Cython is that I want nearly-C-efficient callbacks to give to the library.)

正如我在其他问题中所看到的那样，没有办法用C做到这一点。但是在Cython中，我可以编译为：

There is no way to do this in C, as observed by people in my other question. But in Cython I can compile this:

cdef class Quadratic:
    cdef double q11
    cdef double q12
    cdef double q22
    def __cinit__(self, double a, double b, double c):
        self.q11 = a
        self.q12 = b
        self.q22 = c
    cdef f(self, double x[]):
        return self.q11*x[0]*x[0] + self.q12*x[0]*x[1] + self.q22*x[1]*x[1]

（我尚未尝试使用

我的问题-此评估中是否有Python开销？我希望参数化函数的效率几乎与用C语言编写的函数一样。

My question - is there Python overhead in this evaluation? I would like to have the parameterized function be nearly as efficient as if it were written in C.

如果可能的话，Cython如何实现这一目标？

If that is possible, how does Cython achieve this?

推荐答案

（除非这是Cython的新增功能），此不是有效的Cython代码，并且不能使工作。它对我来说失败，并显示错误

(Unless this is a very new addition to Cython then) this is not valid Cython code and can't be made to work. It fails for me with the error

此处不允许使用C函数定义

C function definition not allowed here

这样做的原因是-正如您已经知道的-这是对C的真正限制：Cython不能生成有效的C代码来创建闭包。原因是C将需要能够在运行时动态生成函数以访问封闭的变量，而在标准C中是不可能的。

The reason for this is that - as you already know - it is a genuine limitation of C: there is no valid C code that Cython could generate to create a closure. The reason being is that C would need to be able to dynamically generate a function at runtime to access the closured variables, and that isn't possible in standard C.

至少有三种方法。至少有两种方法。首先（也是最明智的选择），您更改接口以接受数据指针：

There are (at least) two three ways round this. First (and probably most sensible) you change your interface to accept a data pointer:

double[] minimizer(ObjFun f, void* data)

with

typedef double (*ObjFun)(double x[], void* data)

（其中最小化器将 data 传递给 f 。）如果您使用的是外部库，则不能进行更改，但是如果它不使用类似的东西-这是标准解决方案，我会感到惊讶。

(where minimizer passes data to f.) If you're using an external library then you can't make this change, but I'd be surprised if it didn't use something like this - it's the standard solution.

第二种方法是使用 ctypes 将可调用的Python转换为C函数指针。通过在运行时动态生成机器代码来匹配C调用约定，可以解决该问题（因此可以被禁止使内存可执行的安全功能阻止）。我在此答案。 （我敢肯定我还有另一个更彻底的答案显示出来，但是现在找不到）。

The second way is to use ctypes to convert a Python callable into a C function pointer. This gets round the problem by dynamically generating machine code at runtime to match the C calling convention (and so can be blocked by security features that prohibit making memory executable). I show a basic example of how to do this in the second half of this answer. (I'm sure I have another more thorough answer showing this but I can't find it right now).

使用 ctypes 涉及相当数量的Python开销，这是不可避免的。

Using ctypes involves a reasonable amount of Python overhead, which is unavoidable.

（已编辑添加项）模拟闭包的第三个选项（将数据存储在全局变量中：

(Edited addition) The third option to emulate a closure (somewhat unsatisfactorily) is to store the data in global variables:

# at global scope
q11 = 1.2
q22 = 3.2
q33 = 4.0

cdef double f(double x[]):
    # as before

最大的限制是您一次只能使用 f 的一个版本，因此无法运行并行或类似的东西，但要考虑到可能的约束。

The big limitation is that you can only use one version of f at once, so you can't run things in parallel or anything like that, but given the constraints that might be acceptable.

关于您的cdef类版本：调用它非常有效但是它实际上具有以下特征：

With respect to your cdef class version: calling it is pretty efficient but it effectively has a signature of:

`double (*)(Quadratic*, double [])`

它是您的最小化器函数定义的接口。

so it won't fit the interface defined by your minimizer function.

这篇关于Cython具有“头等功能对象”，其功能包括： -他们的效率如何？他们有Python开销吗？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！