NumPy数组索引和/或加法似乎很慢 [英] Numpy array indexing and/or addition seems slow
问题描述
我一直在对numpy数组进行基准测试，因为当我尝试用脚本中的numpy数组替换python数组时，我得到的结果比预期的要慢.
I was playing around with benchmarking numpy arrays because I was getting slower than expected results when I tried to replace python arrays with numpy arrays in a script.
我知道我想念一些东西，我希望有人可以消除我的无知.
I know I'm missing something, and I was hoping someone could clear up my ignorance.
我创建了两个函数并为其计时
I created two functions and timed them
NUM_ITERATIONS = 1000
def np_array_addition():
np_array = np.array([1, 2])
for x in xrange(NUM_ITERATIONS):
np_array[0] += x
np_array[1] += x
def py_array_addition():
py_array = [1, 2]
for x in xrange(NUM_ITERATIONS):
py_array[0] += x
py_array[1] += x
结果:
np_array_addition: 2.556 seconds
py_array_addition: 0.204 seconds
有什么作用?是什么导致大规模的放缓?我发现如果我使用静态大小的数组numpy至少会达到相同的速度.
What gives? What's causing the massive slowdown? I figured that if I was using statically sized arrays numpy would be at least the same speed.
谢谢！
一直让我感到烦恼的是，numpy数组访问速度很慢，而且我想:嘿，它们只是内存中的数组，对吧，Cython应该解决这个问题！"
It kept bothering me that numpy array access was slow, and I figured "Hey, they're just arrays in memory right? Cython should solve this!"
它做到了.这是我修改后的基准
And it did. Here's my revised benchmark
import numpy as np
cimport numpy as np
ctypedef np.int_t DTYPE_t
NUM_ITERATIONS = 200000
def np_array_assignment():
cdef np.ndarray[DTYPE_t, ndim=1] np_array = np.array([1, 2])
for x in xrange(NUM_ITERATIONS):
np_array[0] += 1
np_array[1] += 1
def py_array_assignment():
py_array = [1, 2]
for x in xrange(NUM_ITERATIONS):
py_array[0] += 1
py_array[1] += 1
我将np_array重新定义为cdef np.ndarray[DTYPE_t, ndim=1]
print(timeit(py_array_assignment, number=3))
# 0.03459
print(timeit(np_array_assignment, number=3))
# 0.00755
这是cython也对python函数进行了优化.纯python中python函数的计时是
That's with the python function also being optimized by cython. The timing for the python function in pure python is
print(timeit(py_array_assignment, number=3))
# 0.12510
17倍加速.当然，这是一个愚蠢的例子，但我认为这是有教育意义的.
A 17x speedup. Sure it's a silly example, but I thought it was educational.
推荐答案
这不是(仅仅是)很慢的加法，而是元素访问的开销，例如:
This is not (just) addition which is slow, it is element access overhead, see for example:
def np_array_assignment():
np_array = np.array([1, 2])
for x in xrange(NUM_ITERATIONS):
np_array[0] = 1
np_array[1] = 1
def py_array_assignment():
py_array = [1, 2]
for x in xrange(NUM_ITERATIONS):
py_array[0] = 1
py_array[1] = 1
timeit np_array_assignment()
10000 loops, best of 3: 178 us per loop
timeit py_array_assignment()
10000 loops, best of 3: 72.5 us per loop
在向量(矩阵)上同时执行时，Numpy运算速度很快.这样的逐个元素的操作很慢.
Numpy is fast with operating on vectors (matrices), when performed on the whole structure at once. Such single element-by-element operations are slow.
使用numpy函数避免循环，从而一次对整个数组进行操作，即:
Use numpy functions to avoid looping, making operations on the whole array at once, i.e.:
def np_array_addition_good():
np_array = np.array([1, 2])
np_array += np.sum(np.arange(NUM_ITERATIONS))
将您的功能与上述功能进行比较的结果非常明显:
The results comparing your functions with the one above are pretty revealing:
timeit np_array_addition()
1000 loops, best of 3: 1.32 ms per loop
timeit py_array_addition()
10000 loops, best of 3: 101 us per loop
timeit np_array_addition_good()
100000 loops, best of 3: 11 us per loop
但是实际上，如果您折叠循环，则可以使用纯python做得很好:
But actually, you can do as good with pure python if you collapse the loops:
def py_array_addition_good():
py_array = [1, 2]
rangesum = sum(range(NUM_ITERATIONS))
py_array = [x + rangesum for x in py_array]
timeit py_array_addition_good()
100000 loops, best of 3: 11 us per loop
总而言之，使用如此简单的操作，使用numpy确实没有任何改善.纯python中的优化代码同样有效.
All in all, with such simple operations there is really no improvement in using numpy. Optimized code in pure python works just as good.
对此有很多疑问，我建议在那里查看一些好的答案:
There were a lot of questions about it and I suggest looking at some good answers there:
这篇关于NumPy数组索引和/或加法似乎很慢的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
