Java OpenCV - 从knnMatch中提取好匹配 [英] Java OpenCV - extracting good matches from knnMatch

问题描述

我正在尝试实现一个非常简单的程序，用于查找两个图像之间的相似性。

I am trying to implement a very simple program for finding similarities between two images.

我正在使用ORB功能检测器和图像描述符执行此任务，我就是使用 knnMatch 识别匹配：

I am using the ORB feature detector and image descriptor for this task and I am identifying the matches using knnMatch:

FeatureDetector detector = FeatureDetector.create(FeatureDetector.ORB);
DescriptorExtractor descriptor = DescriptorExtractor.create(DescriptorExtractor.ORB);
DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

// DETECTION
// first image
Mat img1 = Imgcodecs.imread(path1, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
Mat descriptors1 = new Mat();
MatOfKeyPoint keypoints1 = new MatOfKeyPoint();

detector.detect(img1, keypoints1);
descriptor.compute(img1, keypoints1, descriptors1);

// second image
Mat img2 = Imgcodecs.imread(path2, Imgcodecs.CV_LOAD_IMAGE_GRAYSCALE);
Mat descriptors2 = new Mat();
MatOfKeyPoint keypoints2 = new MatOfKeyPoint();

detector.detect(img2, keypoints2);
descriptor.compute(img2, keypoints2, descriptors2);

// MATCHING
// match these two keypoints sets
List<MatOfDMatch> matches = new ArrayList<MatOfDMatch>();
matcher.knnMatch(descriptors1, descriptors2, matches, 5);

我可以按以下方式绘制匹配项：

I am able to draw the matches as follows:

// DRAWING OUTPUT
Mat outputImg = new Mat();
// this will draw all matches, works fine
Features2d.drawMatches2(img1, keypoints1, img2, keypoints2, matches, outputImg);

// save image
Imgcodecs.imwrite("result.jpg", outputImg);

问题在于匹配太多，而且还包括那些远离的匹配。我似乎无法找到如何只提取好的匹配（超过一些阈值）？有人能指出我正确的方向或将我重定向到一些基本的工作示例吗？我花了好几个小时就好了，似乎迷路了......

The problem is that there are too many matches and it includes also those that are way off. I can't seem to find how to extract only the good matches (exceeding some threshold)? Could someone point me to the right direction or redirect me to some basic working example? I have spent several hours on this and seem to be lost..

我试过看关键点匹配工作两次...... ？ （java opencv）但标准（非knn）匹配使用不同的结构，我无法使其工作。

I tried looking at Keypoint matching just works two times...? (java opencv) but the standard (non-knn) match uses different structures and and I could not make it work.

推荐答案

如其他答案所述，有几种方法可以删除异常值和不匹配。我猜你用匹配而不是 knnMatch 使用其中一些方法找到样本和教程。

As mentioned in other answers, there are several methods to remove outliers and bad matches. I guess you found samples and tutorials with match instead of knnMatch utilizing some of those methods.

所以，你可能知道区别在于 knnMatch 返回<$ c $中的n个最佳匹配c> descriptor2 为 descriptor1 中的每个描述符。这意味着，您将获得匹配列表，而不是匹配列表。我想这就是你遇到问题的原因。

So, as you may know the difference is that knnMatch returns the n-best matches in descriptor2 for each descriptor in descriptor1. Which means, instead of a list of matches you get a list of a list of matches. I guess this is the reason, why you had problems.

使用 knnMatch 的主要优点是你可以执行比率测试。因此，如果从 descriptor1 中的一个描述符到 descriptor2 中的两个最佳描述符的距离相似，则表明存在重复图像中的图案（例如草前栅栏的尖端）。因此，这种匹配不可靠，应予以删除。
（我不确定你为什么要搜索五个最佳匹配 - 你为每个描述符传递5到 knnMatch - 而是搜索两个。）

The main advantage using knnMatch is that you can perform a ratio test. So if the distances from one descriptor in descriptor1 to the two best descriptors in descriptor2 are similar it suggests that there are repetitive patterns in your images (e.g. the tips of a picket fence in front of grass). Thus, such matches aren't reliable and should be removed. (I am not sure why you search for the five best matches - you pass 5 to knnMatch - for each descriptor. Rather search for two.)

如果您现在只想为每个描述符访问最佳匹配，则只需访问子列表的第一个元素。在下文中，您将找到使用RANSAC进行比率测试和单应性估计的示例（我在 knnMatch 之后替换了所有内容）：

If you now want to access the best match just for each descriptor, you just have to access the first element of the "sublists". In the following you'll find as a example a ratio test and a homography estimation using RANSAC (I replaced everything after your knnMatch):

    // ratio test
    LinkedList<DMatch> good_matches = new LinkedList<DMatch>();
    for (Iterator<MatOfDMatch> iterator = matches.iterator(); iterator.hasNext();) {
        MatOfDMatch matOfDMatch = (MatOfDMatch) iterator.next();
        if (matOfDMatch.toArray()[0].distance / matOfDMatch.toArray()[1].distance < 0.9) {
            good_matches.add(matOfDMatch.toArray()[0]);
        }
    }

    // get keypoint coordinates of good matches to find homography and remove outliers using ransac
    List<Point> pts1 = new ArrayList<Point>();
    List<Point> pts2 = new ArrayList<Point>();
    for(int i = 0; i<good_matches.size(); i++){
        pts1.add(keypoints1.toList().get(good_matches.get(i).queryIdx).pt);
        pts2.add(keypoints2.toList().get(good_matches.get(i).trainIdx).pt);
    }

    // convertion of data types - there is maybe a more beautiful way
    Mat outputMask = new Mat();
    MatOfPoint2f pts1Mat = new MatOfPoint2f();
    pts1Mat.fromList(pts1);
    MatOfPoint2f pts2Mat = new MatOfPoint2f();
    pts2Mat.fromList(pts2);

    // Find homography - here just used to perform match filtering with RANSAC, but could be used to e.g. stitch images
    // the smaller the allowed reprojection error (here 15), the more matches are filtered 
    Mat Homog = Calib3d.findHomography(pts1Mat, pts2Mat, Calib3d.RANSAC, 15, outputMask, 2000, 0.995);

    // outputMask contains zeros and ones indicating which matches are filtered
    LinkedList<DMatch> better_matches = new LinkedList<DMatch>();
    for (int i = 0; i < good_matches.size(); i++) {
        if (outputMask.get(i, 0)[0] != 0.0) {
            better_matches.add(good_matches.get(i));
        }
    }

    // DRAWING OUTPUT
    Mat outputImg = new Mat();
    // this will draw all matches, works fine
    MatOfDMatch better_matches_mat = new MatOfDMatch();
    better_matches_mat.fromList(better_matches);
    Features2d.drawMatches(img1, keypoints1, img2, keypoints2, better_matches_mat, outputImg);

    // save image
    Imgcodecs.imwrite("result.jpg", outputImg);

我希望这足以作为一个例子。其他过滤方法可以类似地应用。如果您还有其他问题，请随时询问。

I hope this is sufficient as a example. Other filtering methods could be applied analogously. Do not hesitate to ask, if you have further questions.

编辑：
单应性过滤仅在大多数情况下有效关键点位于场景的同一平面上，如墙等。

The homography filtering is only valid if most of your keypoints are on the same plane in the scene, like a wall etc.

这篇关于Java OpenCV - 从knnMatch中提取好匹配的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！