重塑n维数组的视图而无需使用重塑 [英] reshaping a view of a n-dimensional array without using reshape

问题描述

tl; dr 可以在不使用numpy.reshape的情况下将numpy数组的视图从5x5x5x3x3x3重塑为125x1x1x3x3x3吗?

tl;dr Can I reshape a view of a numpy array from 5x5x5x3x3x3 to 125x1x1x3x3x3 without using numpy.reshape?

我想对一个体积(大小为MxMxM)执行滑动窗口操作(具有不同的步幅).滑动窗口数组可以使用 numpy.lib.stride_tricks.as_strided生成. (如 Benjamin 和 Eickenberg ，并在下面的代码段中进行了演示，该代码段使用使用as_strided的skimage的帮助方法.

I would like to perform a sliding window operation (with different strides) to a volume (size of MxMxM). The sliding window array can be generated with the use of numpy.lib.stride_tricks.as_strided, as previously suggested by Benjamin and Eickenberg, and demonstrated in the below code snippet, which uses a helper method from skimage that uses as_strided.

此辅助方法的输出为我提供了NxNxNxnxnxn的形状，但我希望该形状为N ^ 3x1xnxnxn.虽然我可以使用np.reshape实现此目的，但是如果体积变大(> 100x100x100)，np.reshape就会变慢，我不确定为什么.我以为可以使用as_stride重塑输出，但是numpy崩溃了(下面的代码段).关于如何在不使用np.reshape的情况下如何从helper方法获得的输出视图为N ** 3x1xnxnxn的任何想法?

The output from this helper method gives me a shape of NxNxNxnxnxn, but I'd prefer the shape to be N^3x1xnxnxn. While I can use np.reshape to achieve this, np.reshape is slow if the volume gets large (> 100x100x100), which I'm not sure why. I thought I can use as_stride to reshape the output, but numpy crashes (code snippet below). Any ideas on how I can get a view of the output from the helper method as N**3x1xnxnxn without using np.reshape?

import numpy as np
import skimage
l = 15
s = 3
X = np.ones((l,l,l))      
print('actual shape',X.shape)
view = skimage.util.shape.view_as_blocks(X,(s,s,s))    
print('original view',view.shape)

new_shape = ((l/s)**3,1,1,s,s,s)
print('new view',new_shape)    

view_correct = view.reshape(new_shape)
print(view_correct.shape)
print('coord:','124,0,0,2,2,2','value:',view_correct[124,0,0,2,2,2])

view_incorrect = np.lib.stride_tricks.as_strided(view, shape=new_shape)
print(view_incorrect.shape)
print('coord:','124,0,0,2,2,2','value:',view_incorrect[124,0,0,2,2,2])

推荐答案

我从view_as_blocks中举了一个例子，并尝试了您的重塑风格:

I took an example from view_as_blocks, and tried your style of reshape:

A = np.arange(4*4).reshape(4,4)
B = view_as_blocks(A, block_shape=(2, 2))
print(A.__array_interface__)
print(B.__array_interface__)

C = B.reshape((2*2,2,2))
print(C.__array_interface__)

生产:

{'typestr': '<i4', 'data': (153226600, False), 'shape': (4, 4),
 'descr': [('', '<i4')], 'version': 3, 'strides': None}
{'typestr': '<i4', 'data': (153226600, False), 'shape': (2, 2, 2, 2),
 'descr': [('', '<i4')], 'version': 3, 'strides': (32, 8, 16, 4)}
{'typestr': '<i4', 'data': (150895960, False), 'shape': (4, 2, 2),
 'descr': [('', '<i4')], 'version': 3, 'strides': None}

A和B的data指针是相同的； B是A上的视图.

The data pointer for A and B is the same; B is a view on A.

但是C的指针是不同的.它是副本.这就解释了为什么您的案件要花这么长时间.

But the pointer for C is different. It is a copy. That explains why it takes so long in your case.

我们的做法有所不同:

A = np.arange(4*4).reshape(4,4)
B = view_as_blocks(A, block_shape=(2, 2))
print(A.__array_interface__)
print(B.__array_interface__)

C = B.reshape((2*2,1,2,2))
print(C.__array_interface__)

D = as_strided(B, shape=(2*2,1,2,2))
print(D.__array_interface__)

print(B[1,1,:,:])
print(C[3,0,:,:])
print(D[3,0,:,:])

生产

1254:~/mypy$ python3 skshape.py 
{'strides': None, 'typestr': '<i4', 'version': 3, 
 'data': (154278744, False), 'shape': (4, 4), 'descr': [('', '<i4')]}
{'strides': (32, 8, 16, 4), 'typestr': '<i4', 'version': 3, 
 'data': (154278744, False), 'shape': (2, 2, 2, 2), 'descr': [('', '<i4')]}
{'strides': None, 'typestr': '<i4', 'version': 3, 
 'data': (155705400, False), 'shape': (4, 1, 2, 2), 'descr': [('', '<i4')]}
{'strides': (32, 8, 16, 4), 'typestr': '<i4', 'version': 3, 
 'data': (154278744, False), 'shape': (4, 1, 2, 2), 'descr': [('', '<i4')]}

[[10 11]
 [14 15]]
[[10 11]
 [14 15]]
[[  154561960 -1217783696]
 [         48        3905]]

重新整形会创建一个副本.第二个as_strides返回一个视图，但跨度被拧紧.它正在查看原始数据缓冲区之外的内存(这就是为什么自己大步走很危险的部分原因.)

Again the reshape creates a copy. The 2nd as_strides returns a view, but the striding is screwed up. It is looking at memory outside the original data buffer (that's part of why playing with strides on your own is dangerous).

在我的示例中，查看每个块的第一个角点值

In my example, look at the first corner value of each block

print(B[:,:,0,0])
print(C[:,0,0,0])

[[ 0  2]
 [ 8 10]]
[ 0  2  8 10]

对于B，行增加8，列增加2；反映在(32,8)(4 * 8,4 * 2)步幅上.

For B, the rows increase by 8, columns by 2; that's reflected in the (32,8) (4*8,4*2) striding.

但是在C中，步骤是(2,6,2)-大步前进无法做到.

But in C the steps are (2,6,2) - striding can't do that.

由此我得出结论，没有复制就不可能进行重塑.

From this I conclude that the reshape is impossible without copy.

这篇关于重塑n维数组的视图而无需使用重塑的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！