滚动MAD的数字化版本(平均绝对偏差) [英] Numpy version of rolling MAD (mean absolute deviation)
本文介绍了滚动MAD的数字化版本(平均绝对偏差)的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
如何制作以下MAD函数的滚动版本
How to make a rolling version of the following MAD function
from numpy import mean, absolute
def mad(data, axis=None):
return mean(absolute(data - mean(data, axis)), axis)
此代码是此问题的答案
此刻,我将numpy转换为熊猫,然后应用此函数,然后将结果转换回numpy
At the moment i convert numpy to pandas then apply this function, then convert the result back to numpy
pandasDataFrame.rolling(window=90).apply(mad)
但这在较大的数据帧上效率低下.如何在不循环的情况下为numpy中的相同函数获取滚动窗口并给出相同的结果?
but this is inefficient on larger data-frames. How to get a rolling window for the same function in numpy without looping and give the same result?
推荐答案
这是矢量化的NumPy方法-
Here's a vectorized NumPy approach -
# From this post : http://stackoverflow.com/a/40085052/3293881
def strided_app(a, L, S ): # Window len = L, Stride len/stepsize = S
nrows = ((a.size-L)//S)+1
n = a.strides[0]
return np.lib.stride_tricks.as_strided(a, shape=(nrows,L), strides=(S*n,n))
# From this post : http://stackoverflow.com/a/14314054/3293881 by @Jaime
def moving_average(a, n=3) :
ret = np.cumsum(a, dtype=float)
ret[n:] = ret[n:] - ret[:-n]
return ret[n - 1:] / n
def mad_numpy(a, W):
a2D = strided_app(a,W,1)
return np.absolute(a2D - moving_average(a,W)[:,None]).mean(1)
运行时测试-
In [617]: data = np.random.randint(0,9,(10000))
...: df = pd.DataFrame(data)
...:
In [618]: pandas_out = pd.rolling_apply(df,90,mad).values.ravel()
In [619]: numpy_out = mad_numpy(data,90)
In [620]: np.allclose(pandas_out[89:], numpy_out) # Nans part clipped
Out[620]: True
In [621]: %timeit pd.rolling_apply(df,90,mad)
10 loops, best of 3: 111 ms per loop
In [622]: %timeit mad_numpy(data,90)
100 loops, best of 3: 3.4 ms per loop
In [623]: 111/3.4
Out[623]: 32.64705882352941
巨大的 32x+
可以通过循环的熊猫解决方案加快速度!
Huge 32x+
speedup there over the loopy pandas solution!
这篇关于滚动MAD的数字化版本(平均绝对偏差)的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文