使用 numpy `as_strided` 函数创建任意维度的补丁、瓷砖、滚动或滑动窗口 [英] Using numpy `as_strided` function to create patches, tiles, rolling or sliding windows of arbitrary dimension
问题描述
今天早上花了一段时间寻找一个概括性的问题来指出关于as_strided 和/或 如何制作通用窗口函数.似乎有很多问题 关于如何(安全地)在阵列上创建补丁、滑动窗口、滚动窗口、图块或视图以用于机器学习、卷积、图像处理和/或数值积分.
Spent a while this morning looking for a generalized question to point duplicates to for questions about as_strided and/or how to make generalized window functions. There seem to be a lot of questions on how to (safely) create patches, sliding windows, rolling windows, tiles, or views onto an array for machine learning, convolution, image processing and/or numerical integration.
我正在寻找一个通用函数，它可以接受 window、step 和 axis 参数并返回一个 as_strided 查看任意维度.我将在下面给出我的答案，但我很感兴趣，如果有人可以提出更有效的方法，因为我不确定使用 np.squeeze() 是最好的方法，我不确定我的 assert 语句使函数足够安全，可以写入结果视图，而且我不确定如何处理 axis 不是升序的边缘情况.
I'm looking for a generalized function that can accept a window, step and axis parameter and return an as_strided view for over arbitrary dimensions. I will give my answer below, but I'm interested if anyone can make a more efficient method, as I'm not sure using np.squeeze() is the best method, I'm not sure my assert statements make the function safe enough to write to the resulting view, and I'm not sure how to handle the edge case of axis not being in ascending order.
尽职调查
我能找到的最通用的函数是 sklearn.feature_extraction.image.extract_patches 由@eickenberg 编写(以及明显等效的 skimage.util.view_as_windows)，但这些在网络上没有很好的记录，并且可以不要在比原始数组中更少的轴上做窗口(例如，这个问题 要求在一个轴上有一个特定大小的窗口).通常问题也需要 numpy 唯一的答案.
The most generalized function I can find is sklearn.feature_extraction.image.extract_patches written by @eickenberg (as well as the apparently equivalent skimage.util.view_as_windows), but those are not well documented on the net, and can't do windows over fewer axes than there are in the original array (for example, this question asks for a window of a certain size over just one axis). Also often questions want a numpy only answer.
@Divakar 为一维输入创建了一个通用的 numpy 函数 此处，但更高维的输入需要多加注意.我做了一个简单的骨架3d 输入法上的 2D 窗口，但它不是很可扩展.
@Divakar created a generalized numpy function for 1-d inputs here, but higher-dimension inputs require a bit more care. I've made a bare bones 2D window over 3d input method, but it's not very extensible.
推荐答案
EDIT JAN 2020:将可迭代返回从列表更改为生成器以节省内存.
EDIT JAN 2020: Changed the iterable return from a list to a generator to save memory.
EDIT OCT 2020:将生成器放在一个单独的函数中，因为混合生成器和 return 语句不能直观地工作.
EDIT OCT 2020: Put the generator in a separate function, since mixing generators and return statements doesn't work intiutively.
这是我目前的食谱:
def window_nd(a, window, steps = None, axis = None, gen_data = False):
"""
Create a windowed view over `n`-dimensional input that uses an
`m`-dimensional window, with `m <= n`
Parameters
-------------
a : Array-like
The array to create the view on
window : tuple or int
If int, the size of the window in `axis`, or in all dimensions if
`axis == None`
If tuple, the shape of the desired window. `window.size` must be:
equal to `len(axis)` if `axis != None`, else
equal to `len(a.shape)`, or
1
steps : tuple, int or None
The offset between consecutive windows in desired dimension
If None, offset is one in all dimensions
If int, the offset for all windows over `axis`
If tuple, the steps along each `axis`.
`len(steps)` must me equal to `len(axis)`
axis : tuple, int or None
The axes over which to apply the window
If None, apply over all dimensions
if tuple or int, the dimensions over which to apply the window
gen_data : boolean
returns data needed for a generator
Returns
-------
a_view : ndarray
A windowed view on the input array `a`, or `a, wshp`, where `whsp` is the window shape needed for creating the generator
"""
ashp = np.array(a.shape)
if axis != None:
axs = np.array(axis, ndmin = 1)
assert np.all(np.in1d(axs, np.arange(ashp.size))), "Axes out of range"
else:
axs = np.arange(ashp.size)
window = np.array(window, ndmin = 1)
assert (window.size == axs.size) | (window.size == 1), "Window dims and axes don't match"
wshp = ashp.copy()
wshp[axs] = window
assert np.all(wshp <= ashp), "Window is bigger than input array in axes"
stp = np.ones_like(ashp)
if steps:
steps = np.array(steps, ndmin = 1)
assert np.all(steps > 0), "Only positive steps allowed"
assert (steps.size == axs.size) | (steps.size == 1), "Steps and axes don't match"
stp[axs] = steps
astr = np.array(a.strides)
shape = tuple((ashp - wshp) // stp + 1) + tuple(wshp)
strides = tuple(astr * stp) + tuple(astr)
as_strided = np.lib.stride_tricks.as_strided
a_view = np.squeeze(as_strided(a,
shape = shape,
strides = strides))
if gen_data :
return a_view, shape[:-wshp.size]
else:
return a_view
def window_gen(a, window, **kwargs):
#Same docstring as above, returns a generator
_ = kwargs.pop(gen_data, False)
a_view, shp = window_nd(a, window, gen_data = True, **kwargs)
for idx in np.ndindex(shp):
yield a_view[idx]
一些测试用例:
a = np.arange(1000).reshape(10,10,10)
window_nd(a, 4).shape # sliding (4x4x4) window
Out: (7, 7, 7, 4, 4, 4)
window_nd(a, 2, 2).shape # (2x2x2) blocks
Out: (5, 5, 5, 2, 2, 2)
window_nd(a, 2, 1, 0).shape # sliding window of width 2 over axis 0
Out: (9, 2, 10, 10)
window_nd(a, 2, 2, (0,1)).shape # tiled (2x2) windows over first and second axes
Out: (5, 5, 2, 2, 10)
window_nd(a,(4,3,2)).shape # arbitrary sliding window
Out: (7, 8, 9, 4, 3, 2)
window_nd(a,(4,3,2),(1,5,2),(0,2,1)).shape #arbitrary windows, steps and axis
Out: (7, 5, 2, 4, 2, 3) # note shape[-3:] != window as axes are out of order
这篇关于使用 numpy `as_strided` 函数创建任意维度的补丁、瓷砖、滚动或滑动窗口的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
