使用 numpy 和 numba Python 优化计算 [英] Optimizing Calculations with numpy and numba Python

问题描述

我正在尝试使用 numba 和 numpy 使 python 更快地运行标准偏差函数.然而问题是 for 循环非常慢，我需要替代方案，以便我可以更快地编写代码.我将 numba 迭代到已经存在的 numpy 版本，但是性能没有太大的提升.我原来的 list_ 中有数百万个值，因此计算标准偏差函数需要很长时间.下面的 list_ 函数是一个非常短的 numpy 数组，它是我的问题的一个示例，因为我无法发布原始列表编号.下面函数中的 for 循环计算由下面 list_ 中的变量 number 定义的每第 n 个数字的标准偏差.我如何才能使当前的这个函数运行得更快.

I am trying to make python run standard deviation functions faster with numba and numpy. However the problem is the for loop is very slow and I need alternatives so that I could make the code much faster. I iterated numba to the already existing numpy version however there is not much of a gain in performance. My original list_ has million of values within it thus it is taking a very long time to compute the standard deviation function. The list_ function down below is a very short numpy array that is meant to be an example for my problem as I wont be able to post the original list numbers. The for loop in the function below calculates the standard deviation of every nth number defined by the variable number in the list_ below. How would I be able to make this current function run faster.

import numpy as np
from numba import njit,jit,vectorize

number = 5
list_= np.array([457.334015,424.440002,394.795990,408.903992,398.821014,402.152008,435.790985,423.204987,411.574005,
404.424988,399.519989,377.181000,375.467010,386.944000,383.614990,375.071991,359.511993,328.865997,
320.510010,330.079010,336.187012,352.940002,365.026001,361.562012,362.299011,378.549011,390.414001,
400.869995,394.773010,382.556000])

普通代码:

def std_():
    std = np.array([list_[i:i+number].std() for i in range(0, len(list_)-number)])
    print(std)
std_()

Numba 代码:

jitted_func = njit()(std_)
jitted_func()

性能结果:

推荐答案

您可以以矢量化的方式执行此操作.

You can do this in a vectorised fashion.

def rolling_window(a, window):
    shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
    strides = a.strides + (a.strides[-1],)
    return np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)

def std_():
    std = np.array([list_[i:i+number].std() for i in range(0, len(list_)-number)])
    return std

std1 = np.std(rolling_window(list_, 5), axis=1)
print(np.allclose(std1[:-1], std_()))

给出 True.rolling_window 的代码取自 这个答案.

Gives True. The code for rolling_window has been taken from this answer.

与numba的比较-

import numpy as np
from numba import njit,jit,vectorize

number = 5
list_= np.random.rand(10000)

def rolling_window(a, window):
    shape = a.shape[:-1] + (a.shape[-1] - window + 1, window)
    strides = a.strides + (a.strides[-1],)
    return np.lib.stride_tricks.as_strided(a, shape=shape, strides=strides)

def std_():
    std = np.array([list_[i:i+number].std() for i in range(0, len(list_)-number)])
    return std

%timeit np.std(rolling_window(list_, 5), axis=1)
%%timeit
jitted_func = njit()(std_)
jitted_func()

给予

499 µs ± 3.98 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
106 ms ± 2.87 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

这篇关于使用 numpy 和 numba Python 优化计算的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！