带有上限/下限的Numpy自定义Cumsum函数? [英] Numpy custom Cumsum function with upper/lower limits?

问题描述

我有一个numpy/pandas值列表:

I have a numpy/pandas list of values:

a = np.random.randint(-100, 100, 10000)
b = a/100

我想应用自定义的cumsum函数，但是我还没有找到一种没有循环的方法.自定义函数将累加值设置为1的上限和-1的下限，如果求和的加"超出了这些限制，则加"变为0.

I want to apply a custom cumsum function, but I haven't found a way to do it without loops. The custom function sets an upper limit of 1 and lower limit of -1 for the cumsum values, if the "add" to sum is beyond these limits the "add" becomes 0.

如果总和在-1和1的限制之间，但加"的值将超出限制，则加"将成为-1或1的余数.

In the case that sum is between the limits of -1 and 1 but the "added" value would break beyond the limits, the "added" becomes the remainder to -1 or 1.

这是循环版本:

def cumsum_with_limits(values):
    cumsum_values = []
    sum = 0
    for i in values:
        if sum+i <= 1 and sum+i >= -1: 
            sum += i
            cumsum_values.append(sum)
        elif sum+i >= 1:
            d = 1-sum # Remainder to 1
            sum += d
            cumsum_values.append(sum)
        elif sum+i <= -1:
            d = -1-sum # Remainder to -1
            sum += d
            cumsum_values.append(sum)

    return cumsum_values

有什么方法可以对此向量化?我需要在大型数据集上运行此功能，而性能是我当前的问题.感谢任何帮助！

Is there any way to vectorize this? I need to run this function on large datasets and performance is my current issue. Appreciate any help!

更新:修复了一些代码，并对输出进行了一些澄清: 使用np.random.seed(0)，前6个值是:

Update: Fixed the code a bit, and a little clarification for the outputs: Using np.random.seed(0), the first 6 values are:

b = [0.72, -0.53, 0.17, 0.92, -0.33, 0.95]

预期输出:

o = [0.72, 0.19, 0.36, 1, 0.67, 1]

推荐答案

循环不一定是不可取的.如果性能是一个问题，请考虑numba.在不实质改变逻辑的前提下，性能提高了约330倍:

Loops aren't necessarily undesirable. If performance is an issue, consider numba. There's a ~330x improvement without materially changing your logic:

from numba import njit

np.random.seed(0)
a = np.random.randint(-100, 100, 10000)
b = a/100

@njit
def cumsum_with_limits_nb(values):
    n = len(values)
    res = np.empty(n)
    sum_val = 0
    for i in range(n):
        x = values[i]
        if (sum_val+x <= 1) and (sum_val+x >= -1):
            res[i] = x
            sum_val += x
        elif sum_val+x >= 1:
            d = 1-sum_val # Remainder to 1
            res[i] = d
            sum_val += d
        elif sum_val+x <= -1:
            d = -1-sum_val # Remainder to -1
            res[i] = d
            sum_val += d
    return res

assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb(b)).all()

如果您不介意牺牲某些性能，则可以更简洁地重写此循环:

If you don't mind sacrificing some performance, you can rewrite this loop more succinctly:

@njit
def cumsum_with_limits_nb2(values):
    n = len(values)
    res = np.empty(n)
    sum_val = 0
    for i in range(n):
        x = values[i]
        next_sum = sum_val + x
        if np.abs(next_sum) >= 1:
            x = np.sign(next_sum) - sum_val
        res[i] = x
        sum_val += x
    return res

具有与nb2类似的性能，这是一种替代方法(感谢@jdehesa):

With similar performance to nb2, here's an alternative (thanks to @jdehesa):

@njit
def cumsum_with_limits_nb3(values):
    n = len(values)
    res = np.empty(n)
    sum_val = 0
    for i in range(n):
        x = min(max(sum_val + values[i], -1) , 1) - sum_val
        res[i] = x
        sum_val += x
    return res

性能比较:

assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb(b)).all()
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb2(b)).all()
assert np.isclose(cumsum_with_limits(b), cumsum_with_limits_nb3(b)).all()

%timeit cumsum_with_limits(b)      # 12.5 ms per loop
%timeit cumsum_with_limits_nb(b)   # 40.9 µs per loop
%timeit cumsum_with_limits_nb2(b)  # 54.7 µs per loop
%timeit cumsum_with_limits_nb3(b)  # 54 µs per loop

这篇关于带有上限/下限的Numpy自定义Cumsum函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！