使用OpenCV进行图像拼接时遇到困难 [英] Having some difficulty in image stitching using OpenCV

问题描述

我目前正在Visual Studio 2010上使用OpenCV 2.3.1进行图像拼接，但是遇到了一些麻烦.

I'm currently working on Image stitching using OpenCV 2.3.1 on Visual Studio 2010, but I'm having some trouble.

问题描述 我正在尝试编写一个代码来拼接从几个摄像机(大约3到4个)衍生的多个图像，即该代码应继续执行图像拼接，直到我要求停止为止.

Problem Description I'm trying to write a code for stitching multiple images derived from a few cameras(about 3~4), i,e, the code should keep executing image stitching until I ask it to stop.

以下是我到目前为止所做的: (为简化起见，我将用几句话替换部分代码)

The following is what I've done so far: (For simplification, I'll replace some part of the code with just a few words)

     1.Reading frames(images) from 2 cameras (Currently I'm just working on 2 cameras.)
     2.Feature detection, descriptor calculation (SURF)
     3.Feature matching using FlannBasedMatcher
     4.Removing outliers and calculate the Homography with inliers using RANSAC.
     5.Warp one of both images.

对于步骤5，我在以下线程中遵循了答案，只是更改了一些参数: 在opencv中拼接2张图像

For step 5., I followed the answer in the following thread and just changed some parameters: Stitching 2 images in opencv

但是，结果却很糟糕. 我只是将结果上传到youtube上，当然只有知道链接的人才能看到它.

However, the result is terrible though. I just uploaded the result onto youtube and of course only those who have the link will be able to see it.

http://youtu.be/Oy5z_7LeaMk

我的代码如下所示: (仅显示关键部分)

My code is shown below: (Only crucial parts are shown)

VideoCapture cam1, cam2;
cam1.open(0);
cam2.open(1);

while(1)
{
    Mat frm1, frm2;

    cam1 >> frm1;
    cam2 >> frm2;

   //(SURF detection, descriptor calculation 
   //and matching using FlannBasedMatcher)


    double max_dist = 0; double min_dist = 100;

    //-- Quick calculation of max and min distances between keypoints
    for( int i = 0; i < descriptors_1.rows; i++ )
    { 
        double dist = matches[i].distance;
        if( dist < min_dist ) min_dist = dist;
        if( dist > max_dist ) max_dist = dist;
    }


    (Draw only "good" matches 
    (i.e. whose distance is less than 3*min_dist ))            

    vector<Point2f> frame1;
    vector<Point2f> frame2;

    for( int i = 0; i < good_matches.size(); i++ )
    {
      //-- Get the keypoints from the good matches
      frame1.push_back( keypoints_1[ good_matches[i].queryIdx ].pt );
      frame2.push_back( keypoints_2[ good_matches[i].trainIdx ].pt ); 
    }
    Mat H = findHomography( Mat(frame1), Mat(frame2), CV_RANSAC );
    cout << "Homography: " << H << endl;


    /* warp the image */
    Mat warpImage2;
    warpPerspective(frm2, warpImage2, 
    H, Size(frm2.cols, frm2.rows), INTER_CUBIC); 

    Mat final(Size(frm2.cols*3 + frm1.cols, frm2.rows),CV_8UC3); 

    Mat roi1(final, Rect(frm1.cols, 0, frm1.cols, frm1.rows)); 
    Mat roi2(final, Rect(2*frm1.cols, 0, frm2.cols, frm2.rows)); 

    warpImage2.copyTo(roi2); 
    frm1.copyTo(roi1); 
    imshow("final", final); 

我还应该怎么做才能使缝合效果更好?

What else should I do to make the stitching better?

此外，使单应性矩阵固定而不是继续对其进行计算是否合理? 我的意思是我自己指定两台摄像机之间的角度和位移，以便得出满足我想要的单应性矩阵.

Besides, is it reasonable to make the Homography matrix fixed instead of keeping computing it ? What I mean is to specify the angle and the displacement between the 2 cameras by myself so as to derive a Homography matrix that satisfies what I want.

谢谢. :)

推荐答案

听起来您正在明智地进行此操作，但是如果您可以同时使用两个摄像头，并且它们彼此之间保持静止，则离线校准，只需在线应用转换，将使您的应用程序更高效.

It sounds like you are going about this sensibly, but if you have access to both of the cameras, and they will remain stationary with respect to each other, then calibrating offline, and simply applying the transformation online will make your application more efficient.

需要注意的一点是，您说您正在使用 findHomography 函数.在文档中，此功能:

One point to note is, you say you are using the findHomography function from OpenCV. From the documentation, this function:

Finds a perspective transformation between two planes.

但是，您的点在成像3D场景时并不局限于特定的平面.如果要离线校准，则可以用两个摄像头为棋盘成像，并且可以将检测到的角用于此功能.

However, your points are not restricted to a specific plane as they are imaging a 3D scene. If you wanted to calibrate offline, you could image a chessboard with both cameras, and the detected corners could be used in this function.

或者，您可能希望研究基本面矩阵，该矩阵可以使用

Alternatively, you may like to investigate the Fundamental matrix, which can be calculated with a similar function. This matrix describes the relative position of the cameras, but some work (and a good textbook) will be required to extract them.

如果可以找到它，我强烈建议您阅读Richard Hartley和Andrew Zisserman撰写的《计算机视觉中的多视图几何》一书中的第二部分:双视图几何".详细地.

If you can find it, I would strongly recommend having a look at Part II: "Two-View Geometry" in the book "Multiple View Geometry in computer vision", by Richard Hartley and Andrew Zisserman, which goes through the process in detail.

这篇关于使用OpenCV进行图像拼接时遇到困难的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！