将python回调函数传递给Fortran子例程的开销是多少？ [英] what's the overhead of passing python callback functions to Fortran subroutines?

问题描述

我使用F2PY将一个Fortran 90子例程包装到python中。这里的微妙之处在于，Fortran子程序还将python回调函数作为其参数之一：

  SUBROUTINE f90foo（pyfunc ，a）
 real（kind = 8），intent（in）:: a 
！f2py intent（callback）pyfunc 
外部pyfunc 
！f2py real * 8 y，x 
！f2py y = pyfunc（x）
 
！***调试开始*** 
 print *，'开始循环'
 do i = 1,1000 
p = pyfunc（a）
 end do 
 total = etime（已过去）
 print *，'End：total ='，total，'user ='，elapsed（1） ，'system ='，已过（2）
 stop 
！***调试结束*** 
  

pyfunc 是一个在python代码中定义的python函数。包装工作正常，但运行上面的包装版本，我得到的时间比我使用纯python获得的时间长5倍，如下所示，

  def pythonfoo（k）：
k：标量
返回：标量

 print（'纯Python：开始循环'）
 start = time.time（）
 for xrange（1000）：
p = pyfunc（k）
 elapsed =（time.time（） -  start）
 print（'End：total =％20f'％elapsed）
  

所以，问题是，来自哪些开销？我真的想离开 pyfunc ，因为将它重新编码为纯Fortran函数非常耗时，所以有什么方法可以提高包装器的速度模块？

解决方案

在您发布的代码中， a是双精度浮点数。将它从Fortran传递到Python意味着将Fortran double包装为一个PyFloat对象，这确实有成本。在纯Python版本中，k是一个PyFloat，并且您不支付将其包装1000次的价格。



另一个问题是函数调用本身。从C调用Python函数在性能方面已经很差，但是从Fortran中调用它们会更糟糕，因为还有一层代码将Fortran函数调用约定（关于堆栈等）转换为C函数调用约定。从C调用Python函数时，需要将参数准备为Python对象，通常会创建一个PyTuple对象作为Python函数的* args参数，在模块的表中查找以获取函数指针。 ..

最后但并非最不重要的是：在Fortran和Numpy之间传递2D数组时，您需要关心数组的顺序。在这方面，F2py和numpy可以很聪明，但是如果你的Python代码没有被编写来按照Fortran顺序操作数组，你会得到性能提升。

我不知道pyfunc是做什么的，但如果它接近你发布的内容，用Python编写循环，并且只调用一次函数节省您的时间。如果你需要中间值（ p ），让你的Python函数返回一个包含所有中间值的Numpy数组。

I just wrapped a Fortran 90 subroutine to python using F2PY. The subtlety here is that the Fortran subroutine aslo takes a python call-back function as one of its arguments:

SUBROUTINE f90foo(pyfunc, a)
real(kind=8),intent(in) :: a
!f2py intent(callback) pyfunc
external pyfunc
!f2py real*8 y,x
!f2py y = pyfunc(x)

!*** debug begins***
print *, 'Start Loop'
do i=1,1000
  p = pyfunc(a)
end do
total = etime(elapsed)
print *, 'End: total=', total, ' user=', elapsed(1), ' system=', elapsed(2)
stop
!*** debug ends  ***

The pyfunc is a python function defined elsewhere in my python code. The wrapper works fine, but running the wrapped version above, I got an elapsed time about factor of 5 times longer than what I can get using pure python as follows,

def pythonfoo(k):
    """ k: scalar 
        returns: scalar
    """
    print('Pure Python: Start Loop')
    start = time.time()
    for i in xrange(1000):
        p = pyfunc(k)
    elapsed = (time.time() - start)
    print('End: total=%20f'% elapsed)

So, the question is, what is the overhead coming from? I really want to leave pyfunc as is because it is extremely time-consuming to re-code it into pure fortran function, so is there any way to improve the speed of the wrapper module?

解决方案

In the code you posted, a is double precision float. Passing it from Fortran to Python means wrapping the Fortran double to a PyFloat object, which does have a cost. In the pure Python version, k is a PyFloat and you don't pay the price for wrapping it 1000 times.

Another issue is the function call itself. Calling Python functions from C is already bad performance-wise, but calling them from Fortran is worse, because there is an additional layer of code to transform the Fortran function call conventions (regarding the stack etc.) to C function call conventions. When calling a Python function from C, you need to prepare the arguments as Python objects, generally create a PyTuple object to serve as the *args argument of the Python function, make a lookup in the table of the module to get the function pointer...

Last but not least: you need to take care of the array orders when passing 2D arrays between Fortran and Numpy. F2py and numpy can be smart in that regard, but you'll get performance hits if your Python code is not written to manipulate the arrays in Fortran order.

I don't know what pyfunc is meant to do, but if it is close to what you posted, writing the loop in Python, and calling the function only once will save you time. And if you need the intermediate values (p), let your Python function return a Numpy array with all the intermediate values.

这篇关于将python回调函数传递给Fortran子例程的开销是多少？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！