如何加快numpy中的循环? [英] how to speed up loop in numpy?

问题描述

我想加快这段代码的速度:

I would like to speed up this code :

import numpy as np
import pandas as pd

a = pd.read_csv(path)
closep = a['Clsprc']
delta = np.array(closep.diff())
upgain = np.where(delta >= 0, delta, 0)
downloss = np.where(delta <= 0, -delta, 0)
up = sum(upgain[0:14]) / 14
down = sum(downloss[0:14]) / 14
u = []
d = []
for x in np.nditer(upgain[14:]):
    u1 = 13 * up + x
    u.append(u1)
    up = u1
for y in np.nditer(downloss[14:]):
    d1 = 13 * down + y
    d.append(d1)
    down = d1

以下数据:

0     49.00
1     48.76
2     48.52
3     48.28
...
36785758    13.88
36785759    14.65
36785760    13.19

Name: Clsprc, Length: 36785759, dtype: float64

for循环太慢，我该怎么办才能加快这段代码的速度?我可以对整个操作进行矢量化处理吗?

The for loop is too slow, what can I do to speed up this code? Can I vectorize the entire operation?

推荐答案

简单来说，我认为这是循环的作用:

In simplified terms, I think this is what the loops are doing:

upgain=np.array([.1,.2,.3,.4])    
u=[]
up=1
for x in upgain:                  
    u1=10*up+x
    u.append(u1)
    up=u1

生产:

[10.1, 101.2, 1012.3, 10123.4]

在那里有

np.cumprod([10,10,10,10])，还有为[.1,.2,.3,.4]术语修改的cumsum.但是我无法想到将这些与已编译的numpy函数结合在一起的方法.我们可以编写一个自定义ufunc，并使用其accumulate.或者我们可以在cython(或其他c接口)中编写它.

np.cumprod([10,10,10,10]) is there, plus a modified cumsum for the [.1,.2,.3,.4] terms. But I can't off hand think of a way of combining these with compiled numpy functions. We could write a custom ufunc, and use its accumulate. Or we could write it in cython (or other c interface).

https://stackoverflow.com/a/27912352 建议frompyfunc是编写广义.我不希望节省大量时间，也许会节省2倍.

https://stackoverflow.com/a/27912352 suggests that frompyfunc is a way of writing a generalized accumulate. I don't expect big time savings, maybe 2x.

要使用frompyfunc，请定义:

def foo(x,y):return 10*x+y

循环应用程序(上面)应该是

The loop application (above) would be

def loopfoo(upgain,u,u1):
    for x in upgain:
        u1=foo(u1,x)
        u.append(u1)
    return u

矢量化"版本为:

vfoo=np.frompyfunc(foo,2,1) # 2 in arg, 1 out
vfoo.accumulate(upgain,dtype=object).astype(float)

在先前的SO和 https://github中注明了dtype=object要求. com/numpy/numpy/issues/4155

The dtype=object requirement was noted in the prior SO, and https://github.com/numpy/numpy/issues/4155

In [1195]: loopfoo([1,.1,.2,.3,.4],[],0)
Out[1195]: [1, 10.1, 101.2, 1012.3, 10123.4]

In [1196]: vfoo.accumulate([1,.1,.2,.3,.4],dtype=object)
Out[1196]: array([1.0, 10.1, 101.2, 1012.3, 10123.4], dtype=object)

对于这个小列表，loopfoo更快(3µs v 21µs)

For this small list, loopfoo is faster (3µs v 21µs)

对于100个元素的数组，例如biggain=np.linspace(.1,1,100)，vfoo.accumulate更快:

For a 100 element array, e.g. biggain=np.linspace(.1,1,100), the vfoo.accumulate is faster:

In [1199]: timeit loopfoo(biggain,[],0)
1000 loops, best of 3: 281 µs per loop

In [1200]: timeit vfoo.accumulate(biggain,dtype=object)
10000 loops, best of 3: 57.4 µs per loop

对于更大的biggain=np.linspace(.001,.01,1000)(为避免溢出而使用较小的数字)，将保持5倍速比.

For an even larger biggain=np.linspace(.001,.01,1000) (smaller number to avoid overflow), the 5x speed ratio remains.

这篇关于如何加快numpy中的循环?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！