补丁优先级及其对Criminsi基于样例的修补的影响 [英] Patch priority and its effect on Criminsi's Exemplar Based Inpainting

问题描述

我正在尝试实施基于示例的绘画，如 Criminsi 等.al.保持原始格式(在进行进一步优化之前).

I am trying to implement exemplar based in-painting, as proposed by Criminsi et. al. in its original format (before moving into further optimisations).

我想澄清一些理论上的疑问.

I have a few theoretical doubts which I would like to clarify.

我不确定补丁优先级和填充顺序的确定如何将线性结构传播到用户选择的目标区域中.

I am not sure how the patch priority and the determination of fill-order works to propagate linear structures into the user-selected target region.

据我了解，克里米西建议:

From what I understand, Criminsi suggests the following:

1. 确定填充前沿dΩ.
2. 计算位于填充上的补丁的补丁优先级[C(p)D(p)]
3. 将纹理传播到补丁中-按补丁顺序优先级.
4. 更新置信度值.
5. 从其余未填充部分重新计算填充前沿dΩ.

[重复这5个步骤，直到填充了所有目标区域].

现在，如果对算法的这种理解是正确的(并且如果解释错误，请对我进行纠正)，我有以下疑问:

Now, if this understanding of the algorithm is correct (and, please do correct me if the interpretation is wrong), I have the following doubt:

• 如果我总是只从填充前端填充色块，并且补丁固定在其空间位置上(没有其他补丁填充前)，那么我看不到填充的顺序很重要.通过直接采样源来填充补丁区域，在所有迭代中保持不变.因此，我不了解填充顺序将如何导致任何像素级别传播纹理的差异.从我收集的资料来看，最好的来自源区域的匹配应保持不变，无论命令.我在这里想念什么?

• If I'm always filling patches from the fill front only, and if the patches are fixed in their spatial location (w.r.t other patches on the fill-front), then I don't see how the order of filling-in matters. The patches are filled in by direct sampling of the source region, which stays constant for all iterations. Hence, I do not understand how the order of filling will result in any pixel level difference on the propagated textures. From what I gather, the best match from the source region should remain unaltered, irrespective of the order. What am I missing here?

基于此视频(并假设它是正确的)，补片并非总是仅发生在填充前面.这是如何运作的?这是否意味着我们要计算其中所有补丁的优先级立即定位目标区域，然后根据目标区域进行填充优先事项?然后，填充前确定有何重要性?为什么Criminsi完全确定dΩ?我了解Criminsi的该方法比传统的洋葱皮技术领先一步.但是，究竟如何呢?

Based on this video (and assuming it is correct), the patch filling does not always occur on the fill-front only. How does this work? Does this mean we are to compute the priorities for all patches in the target region at once and go on filling them based on their priorities? Then, how does the fill-front determination matter and why does Criminsi determine dΩ at all? I understand Criminsi's approach was a step ahead of the traditional onion-peel technique. But how, exactly?

在深入研究2-3天的密集编码之前，我真的很想澄清这些问题.

I really wish to clarify these questions before I delve into 2-3 days of intense coding.

请帮助.

谢谢.

++++++++++++++++++++++++++++++++++++++++

+++++++++++++++++++++++++++++++++++++++

在实现此算法时，我还要面对的另一个问题:

One more concern I'm facing while implementing this algorithm:

在传播结构和纹理信息的同时，我直接对源区域进行采样，以使两个补丁中已填充像素的SSD最小化.

While propagating structure and texture information, I'm directly sampling the source region for patches with minimal SSD of the already filled pixels in the two patches.

困扰我的问题是:什么是SSD?我尝试了RGB/HSV值的SSD，但是没有解决.我刚刚开始引用Criminisi对SSD的原始引用，但我只是在想...是否有更简单的方法?

The question bugging me is: SSD of what? I tried SSD of RGB/HSV values but that did not work out. I've just started referencing the original citation of Criminisi for SSD, but I'm just wondering... is there an easier way out?

任何帮助将不胜感激.

推荐答案

先填写:

本文的想法是在信息最可靠的位置修补第一个补丁，从而填充正面.在这个细边框中，补丁的一部分由原始图像(开始时)或先前的结果组成.因此，视频是错误的(顺便说一句效果也不是很好).

The idea of the paper is to inpaint first patches where the information is the most reliable, hence the fill front. In this thin border, a part of the patch is made of the original image (at the beginning) or from the previous result. Thus, the video is wrong (and the results are not so good btw).

如果您不遵循填充前沿方法，则需要一些全局优化过程来确保结果的正确性(请参见流行的

If you do not follow the fill front approach, then you need some global optimization process to ensure the correctness of your result (see the popular graphcut-based textures for example).

补丁传播:

在修补程序上进行修补时，只需复制与当前$ \ Omega $域相交的源修补程序(位于较大的源区域内)的内容.源区域是固定的，而不是您放置的区域，这取决于当前的填充前像素.

When inpainting a patch, you just copy the content of a source patch (that lies inside a larger source region) that intersects the current $\Omega$ domain. The source area is fixed but not the place where you put it, that depends on the current fill front pixel.

优先级功能:

这里的直觉是，由于您要复制源修补程序，因此您应该首先复制纹理最丰富的修补程序.这是为什么?假设您有一个纯黑色(左)和白色(右)的图像，而修复区域位于中间.如果没有优先权，您很可能会先复制纯黑色或白色的补丁.使用优先级功能，您将检测到填充前端中属于黑白区域边界的像素，在正确的位置复制一些半黑色的半白色斑点，然后仅在相关位置添加纯黑/白色斑点

The intuition here is that since you are going to copy source patches, then you should copy the most textured patches first. Why is that? Suppose that you have a purely half black (left and half white (right) image, and the inpainting domain i sin the middle. Without priority, you are likely to copy purely black or white patches first. With priority function, you will detect pixels in the fill front that belong to the border between the black and white areas, copy some half-black half-white patches at the correct positions, and then only add pure black/white patches at relevant positions.

为什么订单很重要，即使前额很小

在以下注释的示例中，我们假设填充优先级为(3、4、5、2、1)的5个像素(1、2、3、4、5).

In the example from the comment below, we suppose that we have a fill front of 5 pixels (1, 2, 3, 4, 5) with priority order (3, 4, 5, 2, 1).

由于填充区域非常小，因此可以推断出要修复的剩余区域非常小.因此，用于填充孔的补丁将相交.

Since the fill front is very small, one can infer that the remaining domain to inpaint is very small. Consequently, the patches used to fill the hole will intersect.

然后，发生的事情是

1. 您复制像素3的源色块.剩余的孔是空的，您可以复制与$ \ Omega $相交的所有色块
2. 您要寻找与像素4匹配的色块.找到后，将其复制到$ \ Omega $ 减与上一个色块(像素3的色块)的交点处.如果交换优先级3和4，则不会保留源补丁的相同部分.
3. 处理下一个填充前像素，等等.

1. you copy a source patch for pixel 3. The remaining hole is empty, you can copy all the patch that intersects $\Omega$
2. you look for a patch that matches pixel 4. Once found, you copy it into $\Omega$ minus the intersection with the previous patch (the one for pixel 3). If you swap the priority of 3 and 4, then you will not retain the same part of the source patches.
3. process the next fill front pixel, etc.

这篇关于补丁优先级及其对Criminsi基于样例的修补的影响的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！