从OpenCV中的grabcut重用模型 [英] Reusing models from grabcut in OpenCV

问题描述

我使用OpenCV示例中的交互式grabcut.py来分割图像并保存前景和背景模型。然后我使用这些模型来分割更多相同类型的图像，因为我不想每次重新训练模型。

I used the interactive grabcut.py from the OpenCV samples to segment an image and saved the foreground and background models. Then I used these models to segment more images of the same kind, as I don't want to retrain the model each time.

运行抓取算法后，掩码全部为零（全部为背景），因此它不会分割任何内容。

After running the grabcut algorithm, the mask is all zeros (all background) and therefore it doesn't segment anything.

from matplotlib import pyplot as plt
import numpy as np
import cv2

img = cv2.imread('usimg1.jpg')
mask = np.zeros(img.shape[:2], np.uint8)
bgdModel = np.load('bgdmodel.npy')
fgdModel = np.load('fgdmodel.npy')

cv2.grabCut(img, mask, None, bgdModel, fgdModel, 5, cv2.GC_EVAL)

mask = np.where((mask==2) | (mask==0), 0, 1).astype('uint8') 
img = img * mask[:, :, np.newaxis]

plt.imshow(img)
plt.show()

我试图用掩码或矩形初始化算法，但这会产生错误，因为模型不是空的（这实际上是我的意思）想要）。

I tried to initialize the algorithm with a mask or a rectangle but this produces an error because the models are not empty (which is what I actually want).

我该怎么做将预先训练过的模型传递给算法，这样每次我分割图像时都不会从头开始重新训练？

How do I have to pass the pre-trained models to the algorithm, such that they are not retrained from scratch each time I'm segmenting an image?

编辑
在rayryeng发表评论后，我实现了以下代码：

EDIT After rayryeng's comment I implemented following code:

cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 0, cv2.GC_INIT_WITH_RECT)
cv2.grabCut(img, mask, rect, bgdModel, fgdModel, 2, cv2.GC_EVAL)

它似乎有效，但第一次调用现在改变了我的模型。在源代码中，它调用 learnGMMs 而不检查是否提供了预训练模型。

It seems to work but the first call now changes my model. In the source code it calls learnGMMs without checking whether a pretrained model is provided.

推荐答案

您使用 cv2.GC_EVAL 时有正确的思路，这样您只需执行分段而无需再次计算模型。
不幸的是，即使使用此标志，这也是OpenCV源本身的限制。如果您在遇到实际的C ++实现时查看 GC_EVAL 条件，它执行此操作接近 cv :: grabcut 方法的结尾。请注意，Python cv2.grabCut 方法是 cv :: grabcut 的包装：

You have the correct line of thinking where you use cv2.GC_EVAL so that you only need to perform the segmentation without having to compute the models again. Unfortunately even when you use this flag, this is a limitation with the OpenCV source itself. If you look at the actual C++ implementation when you encounter the GC_EVAL condition, it does this towards the end of the cv::grabcut method. Note that the Python cv2.grabCut method is a wrapper for cv::grabcut:

if( mode == GC_EVAL )
    checkMask( img, mask );

const double gamma = 50;
const double lambda = 9*gamma;
const double beta = calcBeta( img );

Mat leftW, upleftW, upW, uprightW;
calcNWeights( img, leftW, upleftW, upW, uprightW, beta, gamma );

for( int i = 0; i < iterCount; i++ )
{
    GCGraph<double> graph;
    assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );
    learnGMMs( img, mask, compIdxs, bgdGMM, fgdGMM );
    constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
    estimateSegmentation( graph, mask );
}

你会看到 GC_EVAL 只在代码中遇到一次，那就是检查输入的有效性。罪魁祸首是 learnGMMs 函数。即使您指定了训练模型，这些也会重置，因为对 learnGMMs 的调用会忽略 GC_EVAL 标志，因此这会得到无论你指定什么标志作为输入，都会被调用。

You'll see that GC_EVAL is only encountered once in the code and that's to check the validity of the inputs. The culprit is the learnGMMs function. Even though you specified the trained models, these get reset because the call to learnGMMs ignores the GC_EVAL flag, so this gets called regardless of whatever flag you specify as the input.

受到这篇文章的启发： OpenCV - 带有自定义前景/背景模型的GrabCut ，您可以做的是您必须自己修改OpenCV源，并在循环内部放置一个 if 语句，在调用 learnGMMs GC_EVAL 标志>：

Inspired by this post: OpenCV - GrabCut with custom foreground/background models, what you can do is you'll have to modify the OpenCV source yourself and inside the loop you can place an if statement to check for the GC_EVAL flag prior to calling learnGMMs:

if( mode == GC_EVAL )
    checkMask( img, mask );

const double gamma = 50;
const double lambda = 9*gamma;
const double beta = calcBeta( img );

Mat leftW, upleftW, upW, uprightW;
calcNWeights( img, leftW, upleftW, upW, uprightW, beta, gamma );

for( int i = 0; i < iterCount; i++ )
{
    GCGraph<double> graph;
    assignGMMsComponents( img, mask, bgdGMM, fgdGMM, compIdxs );
    if (mode != GC_EVAL) // New
        learnGMMs( img, mask, compIdxs, bgdGMM, fgdGMM );
    constructGCGraph(img, mask, bgdGMM, fgdGMM, lambda, leftW, upleftW, upW, uprightW, graph );
    estimateSegmentation( graph, mask );
}

这应该可以使用预先训练的模型而无需学习它们在每次迭代中重复一次。一旦你进行了更改，你将不得不重新编译源代码，并且当你使用 cv2.GC_EVAL flag。

This should be able to use the pre-trained models without having to learn them all over again at each iteration. Once you make the change, you'll have to recompile the source again and that should hopefully be able to use your pre-trained models without clearing them when you use the cv2.GC_EVAL flag.

为了将来，我在OpenCV的正式回购中打开了一个问题。希望他们在有空的时候解决这个问题： https://github.com/opencv/ opencv / issues / 9191

For the future, I have opened up a issue on the official repo for OpenCV. Hopefully they'll fix this when they have the time: https://github.com/opencv/opencv/issues/9191

这篇关于从OpenCV中的grabcut重用模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！