OpenCV:如何使用findHomography()/findFundamental()和RANSAC获取内部点 [英] OpenCV: How to get inlier points using findHomography()/findFundamental() and RANSAC

问题描述

OpenCV本身不提供RANSAC函数，或者至少不以您可以调用并使用它的形式提供(例如cv::ransac(...)).能够使用RANSAC的所有功能/方法都有一个启用它的标志.但是，如果您实际上想在估计单应性/基本矩阵之后使用RANSAC计算的其他值，则这并不总是有用的，例如在Octave或类似的软件/点库中创建一个漂亮的图，在该点上应用其他算法剩余的已过滤匹配集等.

OpenCV does not provide a RANSAC-function per se or at least in such a form that you can just call it and be done with it (e.g. cv::ransac(...)). All functions/methods that are able to use RANSAC have a flag that enables it. However this is not always useful if you actually want to do something else with the inliers RANSAC computes after you have estimated a homography/fundamental matrix for example create a nice plot in Octave or similar software/library of the points, apply additional algorithms on the remaining set of filtered matches etc.

匹配两个图像后，将获得一个匹配向量.除此之外，我们当然还有在匹配过程中使用的2组关键点(每个图像一个).使用匹配和关键点，我们创建两个点向量(例如cv::Point2f points)，并将它们传递给findHomography().来自此和<我通过a href ="https://stackoverflow.com/questions/15815304/opencv-c-findhomography-mask-values-含义">此帖子发现了我们如何精确地使用遮罩标记内部该功能.面罩内的每一行都与一个内部/外部相关.但是我无法从我的两点中找出如何使用行索引信息.看着OpenCV的源代码并没有让我太过分.在findFundamental()(类似于findHomography()的签名和遮罩部分)中，他们使用compressPoints()，这似乎以某种方式将我们作为输入的两个集合(源点和目标点)组合为一个.在测试以确定掩膜的性质时，我尝试了2套匹配点(将cv::Keypoints转换为cv::Point2f-标准过程).每组包含300点，因此我们总共有600点.返回的掩码包含300行(值对于此主题而言并不重要).

After matching two images one gets a vector of matches. Along with that we have of course 2 sets of keypoints (one for each image) that were used in the matching process. Using matches and keypoints we create two vectors of points (e.g. cv::Point2f points) and pass these to findHomography(). From this and this posts I discovered how exactly the inliers are marked using a mask, that we pass to that function. Each row inside the mask relates to an inlier/outlier. However I am unable to figure out how to use the row-index information from my two sets of points. Looking at OpenCV's source code didn't get me too far. In findFundamental() (similar to findHomography() when it comes to its signature and the mask-part) they use compressPoints(), which seems to somehow combine the two sets we have as input (source and destination points) into one. While testing in order to determine the nature of the mask I tried 2 sets of matched points (converted cv::Keypoints to cv::Point2f - a standard procedure). Each set contains 300 points so in total we have 600 points. The returned mask contains 300 rows (values are not important for this topic at hand).

在撰写本文时，我发现了答案(请参阅下文)，但还是决定发布此问题，以防有人尽快以紧凑的形式获取此信息.请注意，我们仍然需要支持RANSAC的OpenCV功能之一.因此，如果您有一组要点，但无意计算单应性或基本矩阵，那显然不是这样，我敢说我无法在OpenCV的API中找到任何有用的东西来帮助避免这一障碍，因此您需要使用外部库.

While writing this I discovered the answer (see below) but decided to post this question anyway in case someone needs this information as soon as possible and in compact form. Note that we still need one of OpenCV's function, which support RANSAC. So if you have a set of points but no intention of computing homography or fundamental matrix, this is obviously not the way and I dare say that I was unable to find anything useful in OpenCV's API that can help avoid this obstacle therefore you need to use an external library.

推荐答案

该解决方案实际上很简单.众所周知，如果我们有一个离群值或离群值，则掩码中的每一行都会提供信息.但是，我们有2组点作为输入，因此包含单个值的行如何精确地表示两个点?在考虑这两组点实际上如何出现在findHomography()中时，这种索引的性质出现在我的脑海中(在我的情况下，我正在计算两个图像之间的单应性).这两个集合中的点数相等，这是因为它们是从一对图像之间的匹配中提取出来的简单事实.这意味着遮罩中的一行是两个集合中点的实际索引，也是两个图像的匹配向量中的索引.我已经成功地基于此手动引用了匹配点的一小部分，并且结果符合预期.重要的是，不要使用每个cv::DMatch中引用的关键点来更改匹配项和从中提取的2D点的顺序.在下面，您可以看到一个简单的示例，该示例针对一对Inlier.

The solution is actually quite trivial. As we know each row in our mask gives information if we have an inlier or an outlier. However we have 2 sets of points as input so how exactly does a row containing a single value represent two points? The nature of this sort of indexing appeared in my mind while thinking how actually those two sets of points appear in findHomography() (in my case I was computing the homography between two images). Both sets have equal number of points in them because of the simple fact that they are extracted from the matches between our pair of images. This means that a row in our mask is the actual index of the points in the two sets and also the index in the vector of matches for the two images. I have successfully managed to manually refer to a small subset of matched points based on this and the results are as expected. It is important that you don't alter the order of your matches and the 2D points you have extracted from them using the keypoints referenced in each cv::DMatch. Below you can see a simple example for a single pair of inliers.

for(int i = 0; i < matchesObjectScene.size(); ++i)
{
   // extract points from keypoints based on matches
   pointsObject.push_back(keypointsObject.at(matchesObjectScene.at(i).queryIdx).pt);
   pointsScene.push_back(keypointsScene.at(matchesObjectScene.at(i).trainIdx).pt);
}
// compute homography using RANSAC
cv::Mat mask;
cv::Mat H = cv::findHomography(pointsObject, pointsScene, CV_RANSAC, ransacThreshold, mask);

在上面的示例中，如果我们打印一些内部纸

In the example above if we print some inlier

int maskRow = 10;
std::cout << "POINTS: object(" << pointsObject.at(maskRow).x << "," << pointsObject.at(maskRow).y << ") - scene(" << pointsScene.at(maskRow).x << "," << pointsScene.at(maskRow).y << ")" << std::endl;

，然后再次，但是这次使用我们的关键点(也可以使用提取的2D点来完成)

and then again but this time using our keypoints (can also be done with the extracted 2D points)

std::cout << "POINTS (via match-set): object(" << keypointsObject.at(matchesCurrentObject.at(maskRow).queryIdx).pt.x << "," << keypointsObject.at(matchesCurrentObject.at(maskRow).queryIdx).pt.y << ") - scene(" << keypointsScene.at(matchesCurrentObject.at(maskRow).trainIdx).pt.x << "," << keypointsScene.at(matchesCurrentObject.at(maskRow).trainIdx).pt.y << ")" << std::endl;

我们实际上得到了相同的输出:

we actually get the same output:

POINTS: object(462,199) - sscene(485,49)
POINTS (via match-set): object(462,199) - scene(485,49)

要获取实际的惯常值，我们只需检查掩码中的当前行是否实际包含0或非零值:

To get the actual inlier we simply have to check if the current row in the mask actually contains a 0 or non-zero value:

if((unsigned int)mask.at<uchar>(maskRow))
  // store match or keypoints or points somewhere where you can access them later

这篇关于OpenCV:如何使用findHomography()/findFundamental()和RANSAC获取内部点的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！