Java OpenCV-将knnMatch与findHomography一起使用可显示重复项 [英] Java OpenCV - Using knnMatch with findHomography shows duplicates

问题描述

我是OpenCV Java的新手，我有一个Android应用程序，它将使用ORB FeatureDetector和DescriptorExtractor匹配两个图像.我使用DescriptorMatcher BRUTEFORCE_HAMMING.匹配器一直在工作，而其他时候则显示关键点的重复项.当场景中的图像"太亮或太暗时，它会显示重复的关键点，这不是我想要的.

I am new to OpenCV java and I have an android app that will match two images using ORB FeatureDetector and DescriptorExtractor. I use DescriptorMatcher BRUTEFORCE_HAMMING. All the time the matcher works but other times it shows duplicates of Keypoints. When Image on the Scene is too bright or too dark, it shows duplicate key points which is not what I wanted.

精确匹配的图像:

不正确的图像匹配:

try {
 bmpObjToRecognize = bmpObjToRecognize.copy(Bitmap.Config.ARGB_8888, true);
 bmpScene = bmpScene.copy(Bitmap.Config.ARGB_8888, true);

 img1 = new Mat();
 img2 = new Mat();
 Utils.bitmapToMat(bmpObjToRecognize, img1);
 Utils.bitmapToMat(bmpScene, img2);
 Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGBA2GRAY);
 Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGBA2GRAY);
 Imgproc.equalizeHist(img1, img1);
 Imgproc.equalizeHist(img2, img2);
 detector = FeatureDetector.create(FeatureDetector.ORB);
 descExtractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
 matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);

 keypoints1 = new MatOfKeyPoint();
 keypoints2 = new MatOfKeyPoint();
 descriptors = new Mat();
 dupDescriptors = new Mat();

 detector.detect(img1, keypoints1);
 Log.d("LOG!", "number of query Keypoints= " + keypoints1.size());
 detector.detect(img2, keypoints2);
 Log.d("LOG!", "number of dup Keypoints= " + keypoints2.size());
  // Descript keypoints1
  descExtractor.compute(img1, keypoints1, descriptors);
  descExtractor.compute(img2, keypoints2, dupDescriptors);
  // matching descriptors
  List<MatOfDMatch> knnMatches = new ArrayList<>();
  matcher.knnMatch(descriptors, dupDescriptors, knnMatches, DescriptorMatcher.BRUTEFORCE);
  goodMatches = new ArrayList<>();
  knnMatchesValue = knnMatches.size();
  Log.i("xxx", "xxx match count knnMatches = " + knnMatches.size());
  for (int i = 0; i < knnMatches.size(); i++) {
   if (knnMatches.get(i).rows() > 1) {
    DMatch[] matches = knnMatches.get(i).toArray();
    if (matches[0].distance < 0.89f * matches[1].distance) {
     goodMatches.add(matches[0]);
    }
   }
  }

  // get keypoint coordinates of good matches to find homography and remove outliers using ransac
  List<Point> pts1 = new ArrayList<>();
  List<Point> pts2 = new ArrayList<>();
  for (int i = 0; i < goodMatches.size(); i++) {
   Point destinationPoint = keypoints2.toList().get(goodMatches.get(i).trainIdx).pt;
   pts1.add(keypoints1.toList().get(goodMatches.get(i).queryIdx).pt);
   pts2.add(destinationPoint);
  }

  // convertion of data types - there is maybe a more beautiful way
  Mat outputMask = new Mat();
  MatOfPoint2f pts1Mat = new MatOfPoint2f();
  pts1Mat.fromList(pts1);
  MatOfPoint2f pts2Mat = new MatOfPoint2f();
  pts2Mat.fromList(pts2);

  // Find homography - here just used to perform match filtering with RANSAC, but could be used to e.g. stitch images
  // the smaller the allowed reprojection error (here 15), the more matches are filtered
  Mat Homog = Calib3d.findHomography(pts1Mat, pts2Mat, Calib3d.RANSAC, 15, outputMask, 2000, 0.995);

  // outputMask contains zeros and ones indicating which matches are filtered
  better_matches = new LinkedList<>();
  for (int i = 0; i < goodMatches.size(); i++) {
   if (outputMask.get(i, 0)[0] != 0.0) {
    better_matches.add(goodMatches.get(i));
   }
  }

  matches_final_mat = new MatOfDMatch();
  matches_final_mat.fromList(better_matches);

  imgOutputMat = new Mat();
  MatOfByte drawnMatches = new MatOfByte();
  Features2d.drawMatches(img1, keypoints1, img2, keypoints2, matches_final_mat, 
   imgOutputMat, GREEN, RED, drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);
  bmp = Bitmap.createBitmap(imgOutputMat.cols(), imgOutputMat.rows(), Bitmap.Config.ARGB_8888);
  Imgproc.cvtColor(imgOutputMat, imgOutputMat, Imgproc.COLOR_BGR2RGB);
  Utils.matToBitmap(imgOutputMat, bmp);
  List<DMatch> betterMatchesList = matches_final_mat.toList();
  final int matchesFound = betterMatchesList.size();


} catch (Exception e) {
 e.printStackTrace();
}

我缺少一部分代码吗?

推荐答案

TL; DR使用类BFMatcher及其

TL;DR Use the class BFMatcher and its create method explicitly then your are able set the crosscheck flag to true. This will enable your wanted "vice versa check".

引用knnMatch的OpenCV文档及其标题:

To cite the OpenCV documentation of knnMatch and its header:

从查询集中找到每个描述符的k个最佳匹配.

Finds the k best matches for each descriptor from a query set.

knnMatch(InputArray queryDescriptors, InputArray trainDescriptors, ...)

因此，这意味着可能有多个查询描述符"与训练集"中的同一描述符匹配.它只是给您最好的k，如果查询描述符多于训练描述符，您将不可避免地得到重复项.尤其是，当您几乎没有特征，因此训练图像/集合中没有描述符时(由于缺乏纹理，例如黑色输入)，情况就是如此.

So this means that it is possible that more than one of the "query descriptors" match to the same descriptor in the "training set". It just gives you the k best and if there are more query descriptors than training descriptors you will inevitably get duplicates. Especially, when you almost have no features and therefore descriptors in the training image/set (due to the lack of any texture e.g. your black input), that will be the case.

如果要消除重复项，请将BFMatcher的"crosscheck"标志设置为true.否则(即其他匹配器)，您将需要仔细检查比赛，并根据各自的训练描述符对它们进行分组"，并删除除距离最短以外的所有训练描述符.

If you want to get rid of your duplicates, set the "crosscheck" flag of the BFMatcher to true. Otherwise (i.e. other matcher) you would need to go trough your matches "group" them by the respective training descriptors and remove all but the one with the smallest distance.

这篇关于Java OpenCV-将knnMatch与findHomography一起使用可显示重复项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！