通过不同的"C"以指针数组作为类的函数参数的函数 [英] pass different "C" functions with pointer arrays as the function argument to a class

问题描述

我正在尝试传递不同的函数，这些函数将指针作为python函数的参数.输入函数作为输入参数的一个示例是给定的normal函数:

I am trying to pass different functions which have pointers as arguments to a python function. One example of the input function as input parameter is the given normal function:

Sample.pyx

from cpython cimport array
import cython
import ctypes
cimport numpy as np
cpdef void normal(np.ndarray[ndim=1, dtype=np.float64_t] u, 
                  np.ndarray[ndim=1, dtype=np.float64_t] yu, 
                  np.ndarray[ndim=1, dtype=np.float64_t] ypu):

      cdef int i
      cdef int n=len(u)
      for i in prange(n, nogil=True):
          yu[i]=-u[i]*u[i]*0.5                                                               
          ypu[i]=-u[i]                                                                    
      return                                                     
cdef class _SampleFunc:
     cdef void (*func)(double *, double *, double *)

cdef void sample(int* x, double* hx, double* hxx, void(*func)(double*, double*, double*), int n):
      def int i
      for i from 0 <= i < n:
          func[0](&x[i], &hx[i], &hxx[i])
      return 
cdef class myClass:
     sample_wrapper = _SampleFunc() 
     sample_wrapper.func = Null
     def foo(np.ndarray[ndim=1, dtype=np.float64_t] x,
             np.ndarray[ndim=1, dtype=np.float64_t] hx,
             np.ndarray[ndim=1, dtype=np.float64_t] hxx,
             _SampleFunc sample_func, 
             int k):
         cdef np.ndarray[ndim=1, dtype=np.float64_t] sp
         cdef int num=len(x)
         func = sample_func.func
         assert func is not NULL, "function value is NULL"
         cdef int j
         for j from 0 <= j <k:
             sample(&x[0],&hx[0], &hxx[0], func, num)
             sp[j]=hx[0]
         return sp

test.py

import numpy as np
from sample import *
x = np.zeros(10, float)
hx = np.zeros(10, float)
hpx = np.zeros(10, float)

x[0] = 0
x[1] = 1.0
x[2] = -1.0
def pynormal(x):
    return -x*x*0.5,-x

hx[0], hpx[0] = pynormal(x[0])
hx[1], hpx[1] = pynormal(x[1])
hx[2], hpx[2] = pynormal(x[2])
num=20
ars=myClass()
s=ars.foo( x, hx, hpx, normal, num)

运行test.py代码，我收到此错误:

Running the test.py code I am getting this error:

'ars._SampleFunc' object has no attribute 'func'

我试图为不同的C函数编写包装，这些函数具有三个指针数组作为参数.到目前为止，我的结论是可以用一个类来完成，因为该类可以在python中访问.我想知道如何通过指针数组作为myClass类的参数传递C函数?

I am trying to write a wrapper for different C functions which have three pointer arrays as their argument. My conclusion so far was that it can be done with a class, since the class can be accessible in python. I am wondering how I can pass the C functions with pointer arrays as argument to myClass class?

更新:正常功能

cdef void normal(
                 int n,
                 double* u, 
                 double* yu, 
                 double* ypu
                ):          
      cdef int i          
      for i in prange(n, nogil=True):
          yu[i]=-u[i]*u[i]*0.5                                                               
          ypu[i]=-u[i]                                                                    
      return 

推荐答案

要处理的第一件事是签名cdef void (*func)(double *, double *, double *) 的函数不传递数组长度.您不知道这些数组有多长时间，因此无法安全地访问它们的元素.明智的做法是也更改函数签名以传递长度:

The first thing to deal with is that a function of signature cdef void (*func)(double *, double *, double *) does not pass the array length. You can't know how long these arrays are, and thus you can't safely access their elements. The sensible thing is to change the function signature to pass a length too:

cdef void (*func)(double *, double *, double *, int)

尤其令人困惑的是，您似乎同时在normal和sample中的1D数组的同一轴上进行迭代.我怀疑这不是您想要执行的操作，但我不会尝试解决此问题.

What is extra confusing is that you seem to be iterating over the same axis of a 1D array in both normal and sample. I suspect that isn't what you want to do, but I'm not going attempt to fix that.

本质上，您的问题是您想传递一个可调用的任意Python作为C函数指针.坏消息是Cython无法做到这一点-Python可调用项具有大量与之相关的信息，而C函数指针只是某些可执行内存的地址.因此，C函数指针没有可用空间来保存可调用的Python中的信息.为了使这项工作有效，您需要在运行时生成Python不能做到的代码.

Essentially your problem is that you want to pass an arbitrary Python callable as a C function pointer. The bad news is that Cython can't do it - a Python callable has a significant amount of information associated with it, while a C function pointer is simply the address of some executable memory. Therefore a C function pointer does not have the space available to hold the information in a Python callable. In order to make this work you need to generate code at runtime, which Python can't do.

我建议ctypes标准库模块作为以前解决类似问题的方法，因为它可以

I've recommended the ctypes standard library module as a solution to similar problems previously, since it can create a function pointer from a Python callable. There is a simpler but more limited solution if you only want to call cdef Cython functions.

这是一个最小的示例，演示了如何实现该想法:

Here's a minimal example which demonstrates how to implement the idea:

import numpy as np
import ctypes

ctypedef void (*func_t)(int, double *)

cdef void sample(int n, double* x, func_t f):
    f(n,x)

def call_sample(double[::1] x,
                f):

    def func_wrapper(n, arg1):
        # x is a slightly opaque ctypes type
        # first cast it to a ctypes array of known size
        # and then create a numpy array from that
        arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))
        return f(np.asarray(arg1_as_ctypes_array))


    FTYPE = ctypes.CFUNCTYPE(None, # return type
                             ctypes.c_int, # arguments
                             ctypes.POINTER(ctypes.c_double))
    f_ctypes = FTYPE(func_wrapper) # convert Python callable to ctypes function pointer

    # a rather nasty line to convert to a C function pointer
    cdef func_t f_ptr = (<func_t*><size_t>ctypes.addressof(f_ctypes))[0]

    sample(x.shape[0], &x[0], f_ptr)


def example_function(x):
    # expects a numpy array like object
    print(x)

def test():
    a = np.random.rand(20)
    print(a)
    call_sample(a,example_function)

我意识到ctypes和Cython之间会有一些混乱的转换-这是不可避免的.

I realise that there's some slightly messy conversion between ctypes and Cython - this is unavoidable.

一点解释:我假设您想保持Python接口简单，因此example_function只需要一个类似numpy数组的对象. ctypes传递的函数需要接受许多元素和一个指针，以匹配您的C接口.

A bit of explanation: I'm assuming you want to keep the Python interface simple, hence example_function just takes a numpy array-like object. The function passed by ctypes needs to accept a number of elements and a pointer to match your C interface.

ctypes指针类型(LP_c_double)可以做索引操作(即arg1[5])，因此它可以很好地用于简单用途，但不会在内部存储其长度.将其更改为numpy数组很有帮助(但不是必需的)，因此您可以更广泛地使用它，因此我们创建了包装函数来执行此操作.我们这样做:

The ctypes pointer type (LP_c_double) can do do indexing (i.e. arg1[5]) so it works fine for simple uses, but it doesn't store its length internally. It's helpful (but not essential) to change it to a numpy array so you can use it more generally and thus we create a wrapper function to do this. We do:

arg1_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(arg1.contents))

将其转换为已知长度的ctypes数组，然后

to convert it to a known length ctypes array and then

np.asarray(arg1_as_ctypes_array)

将其转换为numpy数组.这将共享数据而不是进行复制，因此，如果更改它，则原始数据将被更改.由于转换为numpy数组遵循标准模式，因此很容易在call_sample中生成包装函数.

to convert it to a numpy array. This shares the data rather than makes a copy, so if you change it then your original data will be changed. Because the conversion to a numpy array follows a standard pattern it's easy to generate a wrapper function in call_sample.

(在注释中，您询问仅传递double而不是double*时如何进行转换.在这种情况下，您无需执行任何操作，因为ctypes double的行为完全相同像Python类型)

(In the comments you ask how to do the conversion if you're just passing a double, not a double*. In this case you don't have to do anything since a ctypes double behaves exactly like a Python type)

如果确定要传递的函数始终是cdef函数，则可以避免使用ctypes并提出一些更简单的方法.您首先需要使函数签名与指针完全匹配:

If you're certain the functions you want to pass will always be cdef functions then you can avoid ctypes and come up with something a bit simpler. You first need to make the function signature match the pointer exactly:

cdef void normal(int N, double *x): # other parameters as necessary
    cdef double[::1] x_as_mview = <double[:N:1]>x # cast to a memoryview
    # ... etc

然后您应该能够像创建模块级对象一样使用SampleFunc的定义:

You should then be able to use your definition of SampleFunc almost as is to create module level objects:

# in Cython
normal_samplefunc = SampleFunc()
normal_samplefunc.func = &normal

# in Python
s=ars.foo( x, hx, hpx, normal_samplefunc, num)

ars.foo是您编写它的方式(没有ctypes代码):

ars.foo is the way you wrote it (no ctypes code):

func = sample_func.func
# ...
sample(..., func,...)

此代码将运行得更快，但是您希望能够从Python调用normal.

This code will run quicker, but you want be able to call normal from Python.

您在评论中提到，您还希望能够从Python访问normal.对于Python函数和传递给C的接口，您可能需要一个不同的接口，因此我将为这两种用法定义一个单独的函数，但要共享实现:

You mention in the comments that you'd also like the be able to access normal from Python. You're likely to need a different interface for the Python function and the one you pass to C, so I'd define a separate function for both uses, but share the implementation:

def normal(double[::1] u, # ... other arguments
           ):
   # or cpdef, if you really want
   implementation goes here

# then, depending on if you're using ctypes or not:
def normal_ctypes(int n, u # other arguments ...
   ):
   u_as_ctypes_array = (ctypes.c_double*n).from_address(ctypes.addressof(x.contents))
   normal(u_as_ctypes_array, # other arguments
                )

# or
cdef void normal_c(int n, double* u # ...
              ):
   normal(<double[:N:1]>x # ...
          )

这篇关于通过不同的"C"以指针数组作为类的函数参数的函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！