关于图像拼接的一些问题 - 单应性？ [英] Some problems on image stitching - homography?

问题描述

我正在使用Visual Studio 2010上的OpenCV 2.3.1进行图像拼接项目。

I'm working on a project of image stitching using OpenCV 2.3.1 on Visual Studio 2010.

我目前遇到2个问题。

（我的声望不超过10，所以我只能在这篇文章中发布2个超链接。我会在评论区发布另外2个）

(My reputation is not over 10 so I can only post 2 hyperlinks in this post. I'll post another 2 in the comment area)

我按照以下链接中提到的步骤进行操作
拼接2张图片opencv

I followed the steps mentioned in the following link Stitching 2 images in opencv

1. 在两张图片中查找SURF功能并匹配它们


2. 删除异常值RANSAC


3. 计算Homography


4. 将目标图像变形为参考图像

1. Finding SURF features in both images and match them
2. Removing outliers with RANSAC
3. Computing Homography
4. Warping the target image to the reference image

，下图是我目前的结果：

and the picture below is the result I currently have:

使用相机在相同位置但不同方向拍摄两张图像（我使用了三脚架）。

Two images are taken using a camera at the same position but in different direction(I used a tripod).

然后我尝试了另一个测试。这次我仍然使用相同的相机拍摄2张图像。但是，我将相机从原来的位置移开了一点，然后拍了第二张照片。
如图所示，结果非常糟糕：

Then I tried another test. This time I still take 2 images using the same camera. However, I moved my camera a bit from its original position and then I took the second picture. The result is rather terrible as shown:

问题1：**如果2个摄像头位于不同的位置，这是否意味着**，标准的全景拼接技术（基于单应性或相机旋转模型）不起作用？

我试图缝合在不同位置拍摄的图像因为将来我想在不同位置的2个摄像机上实现拼接算法，以便扩大FOV，
有点像这样:(我会在评论中发布图片，plz检查加宽视野）

I tried to stitch images taken at different positions because in the future I would like to implement the stitching algorithm on 2 cameras in different positions so as to widen the FOV, sort of like this:(I'll post the picture in the comment, plz check Widen FOV)

但现在看起来我的方向错了：（。

but now it looks like I'm going the wrong way :(.

我刚刚发现，在算法过程中，功能查找和匹配大部分时间都在进行。

I just found out that during the algorithm, the feature finding and matching takes most of the time.

问题2：我可以只计算2幅图像的某些部分（重叠区域）的特征仍然使用Homography进行转换？
即，不计算整个图像。

Problem 2: Can I just compute features in certain part(Overlap area) of the 2 images and still perform transformation using the Homography? i.e , NOT to compute the whole image.

我想这是因为我认为如果我指定2张图像之间的重叠区域的数量，则不需要计算整个图像中的特征。如果我可以计算和匹配重叠区域中的功能，它应该会大大提高速度。

I think in this way because I think it's not necessary to compute features in the whole image if I specifify the amount of the overlap area between 2 images. If I can just compute and match the features in the overlap area it should greatly increase the speed.

下面显示的第一个代码是整个计算整个功能的代码图片。

The first code shown below is the original code which computes features across the whole images.

    int minHessian = 3000;

    SurfFeatureDetector detector( minHessian );
    vector<KeyPoint> keypoints_1, keypoints_2;

    detector.detect( frm1, keypoints_1 );
    detector.detect( frm2, keypoints_2 );

    //-- Calculate descriptors (feature vectors)  
    SurfDescriptorExtractor extractor; ///

    Mat descriptors_1, descriptors_2;

    extractor.compute( frm1, keypoints_1, descriptors_1 );
    extractor.compute( frm2, keypoints_2, descriptors_2 );

我做了以下事情，试图减少运行整个算法所需的时间：

I did the following thing to try to reduce the time required to run the whole algorithm:

    //detector.detect( frm1(Rect(0.5*frm1.cols,0,0.5*frm1.cols,frm1.rows)), keypoints_1 );
    //detector.detect( frm2(Rect(0,0,0.6*frm2.cols,frm2.rows)), keypoints_2 );

    //-- Calculate descriptors (feature vectors)
    SurfDescriptorExtractor extractor; ///

    Mat descriptors_1, descriptors_2;

    extractor.compute( frm1(Rect(0.5*frm1.cols,0,0.5*frm1.cols,frm1.rows)), keypoints_1, descriptors_1 );
    extractor.compute( frm2(Rect(0,0,0.6*frm2.cols,frm2.rows)), keypoints_2, descriptors_2 );

使用上面的代码，计算时间显着减少，同时给出错误的结果：
（我会在评论中发布图片，plz检查错误结果）

Using the code above, the computation time is significantly decreased while giving a bad result: (I'll post the picture in the comment, plz check Bad Result)

目前卡住了，不知道接下来该做什么。真的希望并且能够提供任何帮助。
谢谢。

Currently stuck and have no idea what to do next. Really hope and aprreciate any help. Thanks.

推荐答案

问题1：
我不能非常肯定，但拼接的问题似乎是由于两张照片之间的相机转换。只有全局单应变换，你无法完美地叠加2幅图像。 Homography仅在以下两种情况下足够：

Problem 1: I can't be very sure, but the problem with the stitching does seem to be due to the camera translation between the 2 pictures. With only a global homography transform, there's no way you can overlay the 2 images perfectly. Homography only suffices in the following 2 cases:

1. 相机经历纯粹的旋转（无翻译）


2. 相机经历一般动作，但场景是平面的

也就是说，你的场景是相当平面的（物体相对于相机的翻译）如果不是瓶子。因此，单应性的近似可能仍然是足够的。你只需要正确地混合图像。要做到这一点，首先需要找到一个地方剪切两幅图像之间差异最小的图像，并应用（例如拉普拉斯）混合。对于安装在汽车顶部的摄像头的问题，这种近似可能仍然是合理的，因此您仍然可以使用单应性模型。

That said, your scene is fairly planar (objects are fairly far away compared to the translation of the camera) if not for the bottle. So an approximation by homography may still be sufficient. You just need to blend the images properly. To do so, you first need to find a place to "cut" the images where there is minimum difference between the 2 images, and apply (e.g. laplacian) blending. For your problem of cameras mounted on top of the car, this approximation may still be reasonable, so you may still be able to use a homography model.

如果单应性正确混合是不够的，您可能需要查看3D重建技术或放松单应性要求的其他方法。文献中有几篇论文涉及镶嵌过程中的视差。然而，这些比基本的单应性拼接要复杂得多。

If homography with proper blending is not sufficient, you may need to look at either 3D reconstruction techniques, or other methods that "relaxes" the homography requirement. There's a couple of papers in the literature that deals with parallax during mosaicking. These are however significantly more complex than the basic homography stitching though.

问题2：
是的，可以这样做，如你很确定重叠的位置。但是，您需要确保此重叠区域不会太小，否则您计算的单应性可能会出现偏差。
您办公室数据集的问题似乎是由于相机翻译，如前所述。

Problem 2: Yes, that can be done, as long you are very sure where the overlap is. However, you need to make sure that this overlapping region is not too small, or else the homography that you compute may be skewed. The problem with your office dataset appears to be due to camera translation, as explained before.

最后，您可能需要调整SURF功能检测/匹配参数有点。特征点似乎略偏低。

Lastly, you might want to adjust your SURF feature detection/matching parameters a bit. The feature points seem to be slightly on the low side.

这篇关于关于图像拼接的一些问题 - 单应性？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！