替代 numpy roll 而不复制数组 [英] Alternative to numpy roll without copying array

问题描述

我正在执行类似以下代码的操作，但我对 np.roll() 函数的性能不满意.我正在对 baseArray 和 otherArray 求和，其中 baseArray 在每次迭代中由一个元素滚动.但是我滚动时不需要 baseArray 的副本，我更喜欢这样的视图，例如，当我将 baseArray 与其他数组相加时，如果 baseArray 滚动了两次，则 baseArray 的第二个元素与第 0 个元素相加otherArray，baseArray 的第三个元素与 otherArray 的第一个元素相加.

I am doing something like following code and I am not happy with performance of np.roll() function. I am summing baseArray and otherArray, where baseArray is rolled by one element in each iteration. But I do not need the copy of the baseArray when I roll it, I would rather prefer a view such that for example when I sum baseArray with other array and if baseArray was rolled twice then the 2nd element of basearray is summed with 0th element of otherArray, 3rd element of baseArray is summed with 1st of otherArray etc.

I.E.实现与 np.roll() 相同的结果，但不复制数组.

I.E. to achieve the same result as with np.roll() but without copying the array.

import numpy as np
from numpy import random
import cProfile

def profile():
    baseArray = np.zeros(1000000)
    for i in range(1000):
        baseArray= np.roll(baseArray,1)
        otherArray= np.random.rand(1000000)
        baseArray=baseArray+otherArray

cProfile.run('profile()')

输出(注意第三行 - 滚动功能):

output (note 3rd row - the roll function):

         9005 function calls in 26.741 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    5.123    5.123   26.740   26.740 <ipython-input-101-9006a6c0d2e3>:5(profile)
        1    0.001    0.001   26.741   26.741 <string>:1(<module>)
     1000    0.237    0.000    8.966    0.009 numeric.py:1327(roll)
     1000    0.004    0.000    0.005    0.000 numeric.py:476(asanyarray)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
     1000   12.650    0.013   12.650    0.013 {method 'rand' of 'mtrand.RandomState' objects}
     1000    0.005    0.000    0.005    0.000 {method 'reshape' of 'numpy.ndarray' objects}
     1000    6.390    0.006    6.390    0.006 {method 'take' of 'numpy.ndarray' objects}
     2000    1.345    0.001    1.345    0.001 {numpy.core.multiarray.arange}
     1000    0.001    0.000    0.001    0.000 {numpy.core.multiarray.array}
     1000    0.985    0.001    0.985    0.001 {numpy.core.multiarray.concatenate}
        1    0.000    0.000    0.000    0.000 {numpy.core.multiarray.zeros}
        1    0.000    0.000    0.000    0.000 {range}

推荐答案

我很确定避免复制是不可能的 由于 numpy 数组在内部表示的方式.一个数组由一个连续的内存地址块和一些元数据组成，这些元数据包括数组维度、项目大小和每个维度的元素之间的间隔(步幅").向前或向后滚动"每个元素需要沿同一维度具有不同的长度步幅，这是不可能的.

I'm pretty sure it's impossible to avoid a copy due to the way in which numpy arrays are represented internally. An array consists of a contiguous block of memory addresses plus some metadata that includes the array dimensions, the item size, and the separation between elements for each dimension (the "stride"). "Rolling" each element forwards or backwards would require having different length strides along the same dimension, which is not possible.

也就是说，您可以避免使用切片索引复制 baseArray 中除一个元素之外的所有元素:

That said, you can avoid copying all but one element in baseArray using slice indexing:

import numpy as np

def profile1(seed=0):
    gen = np.random.RandomState(seed)
    baseArray = np.zeros(1000000)
    for i in range(1000):
        baseArray= np.roll(baseArray,1)
        otherArray= gen.rand(1000000)
        baseArray=baseArray+otherArray
    return baseArray

def profile2(seed=0):
    gen = np.random.RandomState(seed)
    baseArray = np.zeros(1000000)
    for i in range(1000):
        otherArray = gen.rand(1000000)
        tmp1 = baseArray[:-1]               # view of the first n-1 elements
        tmp2 = baseArray[-1]                # copy of the last element
        baseArray[1:]=tmp1+otherArray[1:]   # write the last n-1 elements
        baseArray[0]=tmp2+otherArray[0]     # write the first element
    return baseArray

这些将给出相同的结果:

These will give identical results:

In [1]: x1 = profile1()

In [2]: x2 = profile2()

In [3]: np.allclose(x1, x2)
Out[3]: True

在实践中，性能没有太大差异:

In practice there isn't that much difference in performance:

In [4]: %timeit profile1()
1 loop, best of 3: 23.4 s per loop

In [5]: %timeit profile2()
1 loop, best of 3: 17.3 s per loop

这篇关于替代 numpy roll 而不复制数组的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！