从矩阵中有效提取重叠斑块 [英] Memory efficient extraction of overlapping patches from matrix

问题描述

短篇小说：

这是一个跟进问题：快速方法将图像切割成重叠的补丁并将补丁合并到图像

我如何调整答案中提供的代码，不仅适用于尺寸为x，y的图像，其中像素由浮点数描述，但由一个大小为3,3的矩阵？

How must I adapt the code provided in the answer to work not only on images of size x,y, where a pixel is described by a float, but described by a matrix of size 3,3?

此外，如何调整代码使其返回一个生成器，允许我迭代所有补丁而不必保存所有补丁在记忆中？

Further, how to adapt the code so that it returns a generator allowing me to iterate over all patches without having to save all of them in memory?

长篇故事：

给出形状图像（x ，y），其中每个像素由（3,3）矩阵描述。这可以描述为形状矩阵（x，y，3,3）。
进一步给定目标补丁大小，如（11,11），我想从图像（x，y）中提取所有重叠的补丁。

Given an image of shape (x,y), where each pixel is described by a (3,3) matrix. This can be described as a matrix of shape (x,y,3,3). Further given a target patchsize such as (11,11), I want to extract all overlapping patches from the image (x,y).

请注意，我不想从矩阵x，y，3,3获取所有补丁，而是从图像x，y获取每个像素为矩阵的补丁。

Note that I do not want to get all patches from the matrix x,y,3,3 but from the image x,y where each pixel is a matrix.

我想将这些补丁用于补丁分类算法，有效地迭代所有补丁，提取特征和学习分类器。然而，如果给出巨大的图像和大的补丁大小，则无法在不损害内存限制的情况下执行此操作。

I will want to use these patches for a patch classification algorithm, effectively iterating over all patches, extracting features and learning a classifier. Yet given a huge image and large patchsize, there is no way to perform this operation without hurting the limitation of the memory.

可能的解决方案：

• sklearn.feature_extraction.image.extract_patches_2d 提供目标函数，但由于失败而不适用由于内存限制。 （但是对于带有小补丁大小的给定图像工作正常）


• 快速方法将图像切割成重叠的补丁并将补丁合并到图像。一个好的答案似乎引领了方向，使用步幅而不是实际创建输入图像的副本。然而，我无法调整答案以满足我的需求。

• sklearn.feature_extraction.image.extract_patches_2d provides the target function, yet not applicable since it fails due to memory limitation. (but works fine for the given image with small patchsize)
• Fast Way to slice image into overlapping patches and merge patches to image. A great answer seems to lead the way, using strides and not actually creating a copy of the input image. Yet I have not been able to adapt the answer to fit my needs.

因此问题是：我如何调整此代码以适应适合新的输入数据？

Therefore the question is: How can I adapt this code to fit the new input data?

def patchify(img, patch_shape):
    img = np.ascontiguousarray(img)  # won't make a copy if not needed
    X, Y = img.shape
    x, y = patch_shape
    shape = ((X-x+1), (Y-y+1), x, y) # number of patches, patch_shape
    # The right strides can be thought by:
    # 1) Thinking of `img` as a chunk of memory in C order
    # 2) Asking how many items through that chunk of memory are needed when indices
    #    i,j,k,l are incremented by one
    strides = img.itemsize*np.array([Y, 1, Y, 1])
    return np.lib.stride_tricks.as_strided(img, shape=shape, strides=strides)

推荐答案

虽然您所链接的答案并不正确，但我认为最好不要对这些步骤做出假设数组并简单地重用它已有的任何步幅。它具有额外的好处，即不需要原始数组的副本，即使它不是连续的。对于你的扩展图像形状你会这样做：

While the answer you link is not incorrect, I'd argue it is better to not make assumptions over the strides of the array and simply reuse whatever strides it already has. It has the added benefit of never requiring a copy of the original array, even if it is not contiguous. For your extended image shape you would do:

def patchify(img, patch_shape):
    X, Y, a, b = img.shape
    x, y = patch_shape
    shape = (X - x + 1, Y - y + 1, x, y, a, b)
    X_str, Y_str, a_str, b_str = img.strides
    strides = (X_str, Y_str, X_str, Y_str, a_str, b_str)
    return np.lib.stride_tricks.as_strided(img, shape=shape, strides=strides)

很容易被带走，想要写一些不需要专业化的一般功能特定的阵列维度。如果你觉得有必要去那里，你可以在这个要点中找到一些灵感。

It is easy to get carried away and want to write some more general function that doesn't require specialization for a particular array dimensionality. If you feel the need to go there, you may find some inspiration in this gist.

这篇关于从矩阵中有效提取重叠斑块的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！