Numba jit与scipy [英] Numba jit with scipy

问题描述

所以我想加快我在numba jit 的帮助下编写的程序的速度。但是 jit 似乎与许多scipy函数不兼容，因为它们使用 try ... 除了 ... jit 无法处理的结构（我对吗？）

So I wanted to speed up a program I wrote with the help of numba jit. However jit seems to be not compatible with many scipy functions because they use try ... except ... structures that jit cannot handle (Am I right with this point?)

我想到的一个相对简单的解决方案是复制所需的scipy源代码，并删除 try 除了部分（我已经知道它不会出错，因此 try 部分始终可以正常工作）

A relatively simple solution I came up with is to copy the scipy source code I need and delete the try except parts (I already know that it will not run into errors so the try part will always work anyways)

但是我不喜欢这种解决方案，我不确定它是否会起作用。

However I do not like this solution and I am not sure if it will work.

我的代码结构如下所示

import scipy.integrate as integrate
from scipy optimize import curve_fit
from numba import jit

def fitfunction():
    ...

@jit
def function(x):
    # do some stuff
    try:
        fit_param, fit_cov = curve_fit(fitfunction, x, y, p0=(0,0,0), maxfev=500)
        for idx in some_list:
            integrated = integrate.quad(lambda x: fitfunction(fit_param), lower, upper)
    except:
        fit_param=(0,0,0)
        ...

现在这会导致以下错误：

Now this results in the following error:

LoweringError：对象失败（对象模式后端）

LoweringError: Failed at object (object mode backend)

我认为这是由于 jit 无法处理 try 除外（如果我仅将 jit 放在 curve_fit 和 integrate.quad 部分，并解决我自己的 try 结构除外）



I assume this is due to jit not being able to handle try except (it also does not work if I only put jit on the curve_fit and integrate.quad parts and work around my own try except structure)

import scipy.integrate as integrate
from scipy optimize import curve_fit
from numba import jit

def fitfunction():
    ...

@jit
def integral(lower, upper):
    return integrate.quad(lambda x: fitfunction(fit_param), lower, upper)

@jit
def fitting(x, y, pzero, max_fev)
    return curve_fit(fitfunction, x, y, p0=pzero, maxfev=max_fev)


def function(x):
    # do some stuff
    try:
        fit_param, fit_cov = fitting(x, y, (0,0,0), 500)
        for idx in some_list:
            integrated = integral(lower, upper)
    except:
        fit_param=(0,0,0)
        ...

有没有办法使用 jit 和 scipy.integrate.quad 和 curve_fit ，而无需手动删除所有尝试 除了scipy代码中的结构？

Is there a way to use jit with scipy.integrate.quad and curve_fit without manually deleting all try except structures from the scipy code?

它甚至会加快速度吗？

And would it even speed up the code?

推荐答案

Numba根本不是不是一个通用库来加速代码。使用numba可以更快地解决一类问题（特别是如果您在数组上循环，进行数字运算），但是（1）不支持或（2）只是稍微快一些甚至很多，

Numba simply is not a general-purpose library to speed code up. There is a class of problems that can be solved in a much faster way with numba (especially if you have loops over arrays, number crunching) but everything else is either (1) not supported or (2) only slightly faster or even a lot slower.

[...]甚至可以加快代码速度吗？

[...] would it even speed up the code?

SciPy已经是一个高性能的库，因此在大多数情况下，我希望numba的性能更差（或者很少：稍微好一点）。您可能需要进行一些分析，以了解瓶颈是否确实存在于您所编写的代码中 jit ted，那么您可以得到一些改进。但是我怀疑瓶颈将在SciPy的编译代码中，并且该编译代码可能已经进行了优化（因此，确实不太可能您发现可以仅与该代码竞争的实现）

SciPy is already a high-performance library so in most cases I would expect numba to perform worse (or rarely: slightly better). You might do some profiling to find out if the bottleneck is really in the code that you jitted, then you could get some improvements. But I suspect the bottleneck will be in the compiled code of SciPy and that compiled code is probably already heavily optimized (so it's really unlikely that you find an implementation that could "only" compete with that code).

是否可以在scipy.integrate.quad和curve_fit中使用jit，而无需从scipy代码中手动删除除结构以外的所有try？

Is there a way to use jit with scipy.integrate.quad and curve_fit without manually deleting all try except structures from the scipy code?

正如您正确地假定 try 和 except 一样，

As you correctly assumed try and except is simply not supported by numba at this time.

2.6.1。语言

2.6.1.1。构造

Numba努力支持尽可能多的Python语言，但是一些语言功能在Numba编译的函数中不可用。当前不支持以下Python语言功能：

2.6.1. Language

2.6.1.1. Constructs

Numba strives to support as much of the Python language as possible, but some language features are not available inside Numba-compiled functions. The following Python language features are not currently supported:

[...]

• 异常处理（ try .. 除了， try .. 最后）

• Exception handling (try .. except, try .. finally)

所以这里的答案是否。

这篇关于Numba jit与scipy的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！