OpenCV：基于高斯混合模型的颜色提取 [英] OpenCV: color extraction based on Gaussian mixture model
问题描述
我试图使用opencv EM算法做颜色提取。我使用基于opencv文档中的示例的以下代码：
cv :: Mat capturedFrame（height，width，CV_8UC3）;
int i，j;
int nsamples = 1000;
cv :: Mat samples（nsamples，2，CV_32FC1）;
cv :: Mat labels;
cv :: Mat img = cv :: Mat :: zeros（height，height，CV_8UC3）;
img = capturedFrame;
cv :: Mat sample（1，2，CV_32FC1）;
CvEM em_model;
CvEMParams params;
samples = samples.reshape（2，0）;
for（i = 0; i
//来自训练样本
cv :: Mat samples_part = samples.rowRange i * nsamples / N，（i + 1）* nsamples / N）;
cv :: Scalar mean（（（i％N）+1）* img.rows /（N1 + 1），（（i / N1）+1）* img.rows / +1））;
cv :: Scalar sigma（30,30）;
cv :: randn（samples_part，mean，sigma）;
}
samples = samples.reshape（1,0）;
//初始化模型参数
params.covs = NULL;
params.means = NULL;
params.weights = NULL;
params.probs = NULL;
params.nclusters = N;
params.cov_mat_type = CvEM :: COV_MAT_SPHERICAL;
params.start_step = CvEM :: START_AUTO_STEP;
params.term_crit.max_iter = 300;
params.term_crit.epsilon = 0.1;
params.term_crit.type = CV_TERMCRIT_ITER | CV_TERMCRIT_EPS;
//集合数据
em_model.train（samples，Mat（），params，& labels）;
cv :: Mat probs;
probs = em_model.getProbs（）;
cv :: Mat权重;
weights = em_model.getWeights（）;
cv :: Mat modelIndex = cv :: Mat :: zeros（img.rows，img.cols，CV_8UC3）;
for（i = 0; i {
for（j = 0; j {
sample.at< float>（0）=（float）j;
sample.at< float>（1）=（float）i;
int response = cvRound（em_model.predict（sample））;
modelIndex.data [modelIndex.cols * i + j] = response;
}
}
p>
首先，我要提取每个模型，这里共有五个，然后将这些对应的像素值存储在五个不同的矩阵中。在这种情况下，我可以分别有五种不同的颜色。这里我只得到了他们的索引，有没有办法实现他们对应的颜色呢？为了使它容易，我可以从找到基于这五个GMM的主色。
其次，这里我的示例数据点为100，大约需要3秒。但我想在不超过30毫秒内做所有这些事情。我知道OpenCV背景提取，这是使用GMM，执行真的很快，低于20毫秒，这意味着，必须有一个方法，我在30毫秒内为所有600x800 = 480000像素做所有这些。我发现 predict 函数是最耗时的。
第一个问题：
为了做颜色提取，你首先需要训练EM与您的输入像素。之后，你再次循环所有的输入像素，并使用predict（）来分类每个。我附上一个小例子，利用EM来进行基于颜色的前景/背景分离。它显示如何提取每个高斯的主色（平均值）以及如何访问原始像素颜色。
#include < opencv2 / opencv.hpp>
int main（int argc，char ** argv）{
cv :: Mat source = cv :: imread（test.jpg）;
// ouput images
cv :: Mat meanImg（source.rows，source.cols，CV_32FC3）;
cv :: Mat fgImg（source.rows，source.cols，CV_8UC3）;
cv :: Mat bgImg（source.rows，source.cols，CV_8UC3）;
//将输入图像转换为float
cv :: Mat floatSource;
source.convertTo（floatSource，CV_32F）;
//现在将浮动图像转换为列向量
cv :: Mat samples（source.rows * source.cols，3，CV_32FC1）;
int idx = 0;
for（int y = 0; y
for（int x = 0; x
}
}
//我们只需要2个集群
cv :: EMParams params（2）;
cv :: ExpectationMaximization em（samples，cv :: Mat（），params）;
//两个主导颜色
cv :: Mat means = em.getMeans（）;
//两个主要颜色的权重
cv :: Mat weights = em.getWeights（）;
//我们将前景定义为具有最大权重的优势色彩
const int fgId = weights.at< float>（0）>重量。浮子（1）？ 0：1;
//现在分类每个源像素
idx = 0;
for（int y = 0; y< source.rows; y ++）{
for（int x = 0; x< source.cols; x ++）{
// classify
const int result = cvRound（em.predict（samples.row（idx ++），NULL））;
//得到相应的平均值（主色）
const double * ps = means.ptr< double>（result，0）;
//将相应的平均值设置为平均值图像
float * pd = meanImg.ptr< float>（y，x）;
// float图像需要在[0..1]范围
pd [0] = ps [0] / 255.0;
pd [1] = ps [1] / 255.0;
pd [2] = ps [2] / 255.0;
//设置前台或后台
if（result == fgId）{
fgImg.at< cv :: Point3_< uchar> >（y，x，0）= source.at< cv :: Point3_< uchar> >（y，x，0）;
} else {
bgImg.at< cv :: Point3_< uchar> >（y，x，0）= source.at< cv :: Point3_< uchar> >（y，x，0）;
}
}
}
cv :: imshow（Means，meanImg）;
cv :: imshow（Frontground，fgImg）;
cv :: imshow（Background，bgImg）;
cv :: waitKey（0）;
return 0;
}
我已经测试了下面的图像的代码，它执行相当不错。
>
第二个问题：
我注意到集群的最大数量对性能。因此，最好将其设置为非常保守的值，而不是将其留空或将其设置为样例数，如在您的示例中。此外，文档提到了迭代过程以重复优化具有较少约束参数的模型。也许这给你一些加速。要阅读更多内容，请查看为train（）提供的示例代码中的文档此处 a>。
I am trying to use opencv EM algorithm to do color extraction.I am using the following code based on example in opencv documentation:
cv::Mat capturedFrame ( height, width, CV_8UC3 );
int i, j;
int nsamples = 1000;
cv::Mat samples ( nsamples, 2, CV_32FC1 );
cv::Mat labels;
cv::Mat img = cv::Mat::zeros ( height, height, CV_8UC3 );
img = capturedFrame;
cv::Mat sample ( 1, 2, CV_32FC1 );
CvEM em_model;
CvEMParams params;
samples = samples.reshape ( 2, 0 );
for ( i = 0; i < N; i++ )
{
//from the training samples
cv::Mat samples_part = samples.rowRange ( i*nsamples/N, (i+1)*nsamples/N);
cv::Scalar mean (((i%N)+1)*img.rows/(N1+1),((i/N1)+1)*img.rows/(N1+1));
cv::Scalar sigma (30,30);
cv::randn(samples_part,mean,sigma);
}
samples = samples.reshape ( 1, 0 );
//initialize model parameters
params.covs = NULL;
params.means = NULL;
params.weights = NULL;
params.probs = NULL;
params.nclusters = N;
params.cov_mat_type = CvEM::COV_MAT_SPHERICAL;
params.start_step = CvEM::START_AUTO_STEP;
params.term_crit.max_iter = 300;
params.term_crit.epsilon = 0.1;
params.term_crit.type = CV_TERMCRIT_ITER|CV_TERMCRIT_EPS;
//cluster the data
em_model.train ( samples, Mat(), params, &labels );
cv::Mat probs;
probs = em_model.getProbs();
cv::Mat weights;
weights = em_model.getWeights();
cv::Mat modelIndex = cv::Mat::zeros ( img.rows, img.cols, CV_8UC3 );
for ( i = 0; i < img.rows; i ++ )
{
for ( j = 0; j < img.cols; j ++ )
{
sample.at<float>(0) = (float)j;
sample.at<float>(1) = (float)i;
int response = cvRound ( em_model.predict ( sample ) );
modelIndex.data [ modelIndex.cols*i + j] = response;
}
}
My question here is:
Firstly, I want to extract each model, here totally five, then store those corresponding pixel values in five different matrix. In this case, I could have five different colors seperately. Here I only obtained their indexes, is there any way to achieve their corresponding colors here? To make it easy, I can start from finding the dominant color based on these five GMMs.
Secondly, here my sample datapoints are "100", and it takes about nearly 3 seconds for them. But I want to do all these things in no more than 30 milliseconds. I know OpenCV background extraction, which is using GMM, performs really fast, below 20ms, that means, there must be a way for me to do all these within 30 ms for all 600x800=480000 pixels. I found predict function is the most time consuming one.
First Question:
In order to do color extraction you first need to train the EM with your input pixels. After that you simply loop over all the input pixels again and use predict() to classify each of them. I've attached a small example that utilizes EM for foreground/background separation based on colors. It shows you how to extract the dominant color (mean) of each gaussian and how to access the original pixel color.
#include <opencv2/opencv.hpp>
int main(int argc, char** argv) {
cv::Mat source = cv::imread("test.jpg");
//ouput images
cv::Mat meanImg(source.rows, source.cols, CV_32FC3);
cv::Mat fgImg(source.rows, source.cols, CV_8UC3);
cv::Mat bgImg(source.rows, source.cols, CV_8UC3);
//convert the input image to float
cv::Mat floatSource;
source.convertTo(floatSource, CV_32F);
//now convert the float image to column vector
cv::Mat samples(source.rows * source.cols, 3, CV_32FC1);
int idx = 0;
for (int y = 0; y < source.rows; y++) {
cv::Vec3f* row = floatSource.ptr<cv::Vec3f > (y);
for (int x = 0; x < source.cols; x++) {
samples.at<cv::Vec3f > (idx++, 0) = row[x];
}
}
//we need just 2 clusters
cv::EMParams params(2);
cv::ExpectationMaximization em(samples, cv::Mat(), params);
//the two dominating colors
cv::Mat means = em.getMeans();
//the weights of the two dominant colors
cv::Mat weights = em.getWeights();
//we define the foreground as the dominant color with the largest weight
const int fgId = weights.at<float>(0) > weights.at<float>(1) ? 0 : 1;
//now classify each of the source pixels
idx = 0;
for (int y = 0; y < source.rows; y++) {
for (int x = 0; x < source.cols; x++) {
//classify
const int result = cvRound(em.predict(samples.row(idx++), NULL));
//get the according mean (dominant color)
const double* ps = means.ptr<double>(result, 0);
//set the according mean value to the mean image
float* pd = meanImg.ptr<float>(y, x);
//float images need to be in [0..1] range
pd[0] = ps[0] / 255.0;
pd[1] = ps[1] / 255.0;
pd[2] = ps[2] / 255.0;
//set either foreground or background
if (result == fgId) {
fgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0);
} else {
bgImg.at<cv::Point3_<uchar> >(y, x, 0) = source.at<cv::Point3_<uchar> >(y, x, 0);
}
}
}
cv::imshow("Means", meanImg);
cv::imshow("Foreground", fgImg);
cv::imshow("Background", bgImg);
cv::waitKey(0);
return 0;
}
I've tested the code with the following image and it performs quite good.
Second Question:
I've noticed that the maximum number of clusters has a huge impact on the performance. So it's better to set this to a very conservative value instead of leaving it empty or setting it to the number of samples like in your example. Furthermore the documentation mentions an iterative procedure to repeatedly optimize the model with less-constrained parameters. Maybe this gives you some speed-up. To read more please have a look at the docs inside the sample code that is provided for train() here.
这篇关于OpenCV：基于高斯混合模型的颜色提取的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
