高效地显示相邻像素 [英] Efficient inpaint with neighbouring pixels

问题描述

我正在实施一个简单的算法来对受损的图像进行绘画。我有一个预定义的掩码，指定需要修复的区域。我的策略是从蒙版区域的边界开始，用相邻非零像素的中心平均值绘制每个像素，重复直到没有未知像素为止。

I am implementing a simple algorithm to do in-painting on a "damaged" image. I have a predefined mask that specifies the area which needs to be fixed. My strategy is to start at the border of the masked area and in-paint each pixel with the central mean of its neighboring non-zero pixels, repeating until there's no unknown pixels left.

function R = inPainting(I, mask)
H = [1 2 1; 2 0 2; 1 2 1];
R = I;

n = 1;
[row,col,~] = find(~mask); %Find zeros in mask (area to be inpainted)
unknown = horzcat(row, col)';
while size(unknown,2) > 0
    new_unknown = [];
    new_R = R;
    for u = unknown
        r = u(1);
        c = u(2);
        nb = R(max((r-n), 1):min((r+n), end), max((c-n),1):min((c+n),end));
        nz = nb~=0;
        nzs = sum(nz(:));

        if nzs ~= 0 %We have non-zero neighbouring pixels. In-paint with average.
            new_R(r,c) = sum(nb(:)) / nzs;
        else
            new_unknown = horzcat(new_unknown, u);
        end
    end
    unknown = new_unknown;
    R = new_R;
end

这很有效，但效率不高。是否可以使用大多数矩阵运算来矢量化这种方法？有人知道一种更有效的方法来实现这个算法吗？

This works well, but it's not very efficient. Is it possible to vectorize such an approach, using mostly matrix operations? Does someone know of a more efficient way to implement this algorithm?

推荐答案

如果我理解你的问题陈述，你会得到一个面具并且您希望使用掩模中每个像素周围的邻域像素的平均值填充此蒙版中的这些像素。另一个约束是图像被定义为使得属于相同空间位置中的掩模的任何像素在该掩模中为零。你是从面具的边界开始，并将信息传播到面具的内部。鉴于此算法，遗憾的是，您无法使用标准过滤技术执行此操作，因为当前时间步长取决于之前的时间步长。

If I understand your problem statement, you are given a mask and you wish to fill in these pixels in this mask with the mean of the neighbourhood pixels that surround each pixel in the mask. Another constraint is that the image is defined such that any pixels that belong to the mask in the same spatial locations are zero in this mask. You are starting from the border of the mask and are propagating information towards the innards of the mask. Given this algorithm, there is unfortunately no way you can do this with standard filtering techniques as the current time step is dependent on the previous time step.

图像过滤机制，如 imfilter 或 conv2 因为这种依赖性而无法在这里工作。

Image filtering mechanisms, like imfilter or conv2 can't work here because of this dependency.

因此，我能做什么do可以帮助你加快循环内部的速度，希望这会让你整体加快速度。我将向您介绍一个名为 im2col <的函数/ code> 。这是来自图像处理工具箱，并且鉴于您可以使用 imfilter ，我们可以使用此功能。

As such, what I can do is help you speed up what is going on inside your loop and hopefully this will give you some speed up overall. I'm going to introduce you to a function called im2col. This is from the image processing toolbox, and given that you can use imfilter, we can use this function.

im2col 创建一个二维矩阵，使每列都是一个像素邻域展开到一个向量中。工作原理是列主要顺序中的每个像素邻域都被抓取，因此我们在图像的左上角得到一个像素邻域，然后向下移动一行，然后向下移动，直到我们到达最后一行。然后我们移动一列并重复相同的过程。对于我们拥有的每个像素邻域，它将展开到单个向量中，输出将是 MN x K 矩阵，其中您的邻域大小 M x N 对于每个像素邻域，并且 K 邻域。

im2col creates a 2D matrix such that each column is a pixel neighbourhood unrolled into a single vector. How it works is that each pixel neighbourhood in column major order is grabbed, so we get a pixel neighbourhood at the top left corner of the image, then move down one row, and another row and we keep going until we reach the last row. We then move one column over and repeat the same process. For each pixel neighbourhood that we have, it gets unrolled into a single vector, and the output would be a MN x K matrix where you have a neighbourhood size of M x N for each pixel neighbourhood and there are K neighbourhoods.

因此，在循环的每次迭代中，我们可以将当前的修复图像的像素邻域展开为单个向量，确定哪些像素邻域非零，并从那里确定每个所选像素邻域有多少零值。之后，我们计算这些非零列的平均值，忽略零元素。完成后，我们更新图像并移至下一次迭代。

Therefore, at each iteration of your loop, we can unroll the current inpainted image's pixel neighbourhoods into single vectors, determine which pixel neighborhoods are non-zero and from there, determine how many zero values there are for each of these selected pixel neighbourhood. After, we compute the mean for these non-zero columns disregarding the zero elements. Once we're done, we update the image and move to the next iteration.

我们首先需要做的是用1像素填充图像边界，以便我们能够抓住超出图像边界的邻域。您可以使用 padarray ，也来自图像处理工具箱。

What we're going to need to do first is pad the image with a 1 pixel border so that we're able to grab neighbourhoods that extend beyond the borders of the image. You can use padarray, also from the image processing toolbox.

因此，我们可以这样做：

Therefore, we can simply do this:

function R = inPainting(I, mask)
R = double(I); %// For precision

n = 1;
%// Change - column major indices
unknown = find(~mask); %Find zeros in mask (area to be inpainted)  

%// Until we have searched all unknown pixels
while numel(unknown) ~= 0    
    new_R = R;

    %// Change - take image at current iteration and 
    %// create columns of pixel neighbourhoods
    padR = padarray(new_R, [n n], 'replicate');
    cols = im2col(padR, [2*n+1 2*n+1], 'sliding');  

    %// Change - Access the right pixel neighbourhoods
    %// denoted by unknown       
    nb = cols(:,unknown);

    %// Get total sum of each neighbourhood
    nbSum = sum(nb, 1);

    %// Get total number of non-zero elements per pixel neighbourhood
    nzs = sum(nb ~= 0, 1);

    %// Replace the right pixels in the image with the mean
    new_R(unknown(nzs ~= 0)) = nbSum(nzs ~= 0) ./ nzs(nzs ~= 0);

    %// Find new unknown pixels to look at
    unknown = unknown(nzs == 0);

    %// Update image for next iteration
    R = new_R;
end

%// Cast back to the right type
R = cast(R, class(I));

这篇关于高效地显示相邻像素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！