基于4个共面点计算照相机姿态与单应性矩阵 [英] Computing camera pose with homography matrix based on 4 coplanar points

问题描述

我在一个表示四边形（不一定是正方形或矩形）的视频（或图像）中有4个共面点，我希望能够在它们的顶部显示一个虚拟立方体，

由于点是共面的，我可以计算单位平方的角之间的单应性（即[0,0] [0， 1] [1,0] [1,1]）和四边形的视频坐标。

从这个单应性，我应该能够计算正确的相机姿态，即[R | t]，其中R是3x3旋转矩阵，t是3x1平移向量

我已经阅读了许多解决方案（其中一些在SO），并尝试实现它们，但他们似乎只在一些简单的情况下（例如当视频四边形是一个正方形），但在大多数情况下不工作。

这里是我尝试的方法相同的原理，只有翻译的计算略有不同）。令K是来自相机的内在矩阵，H是单应性。我们计算：

  A = K-1 * H 
  

令a1，a2，a3为A的列向量，r1，r2，r3为旋转矩阵R的列向量。

  r1 = a1 / || a1 || 
 r2 = a2 / || a2 || 
 r3 = r1 x r2 
t = a3 / sqrt（|| a1 || * || a2 ||）
  

问题是这在大多数情况下不工作。为了检查我的结果，我将R和t与OpenCV的solvePnP方法（使用以下3D点[0,0,0] [0,1,0] [1,0,0] [1,1] ，0]）。

由于我以相同的方式显示多维数据集，我注意到在每种情况下solvePnP提供正确的结果，错误。

在理论上，由于我的点是共面的，可以从单应性计算姿势，但是我找不到正确的方法来计算H

对我做错了什么洞见？

尝试@ Jav_Rock的方法后编辑

Hi Jav_Rock，非常感谢你的答案，我试着你的方法（以及许多其他人），似乎是或多或少确定。
然而，我仍然碰巧在计算基于4共面点的姿势时有一些问题。为了检查结果，我与solvePnP（由于迭代重投影误差最小化方法将更好）的结果进行比较。

这里是一个例子： p>

黄色立方体：Solve PNP
黑色立方体：Jav_Rock的技术
青色（和紫色）立方体：一些其他技术给出完全相同的结果

正如你所看到的，黑色的立方体或多或少是好的，但看起来并不合理，虽然矢量似乎正交。

EDIT2：我在计算它之后归一化v3（为了强制正交性），它似乎也解决了一些问题。

解决方案

如果您有自己的人口统计学，可以用以下方法计算相机姿势：

  void cameraPoseFromHomography（const Mat& H，Mat& pose）
 {
 pose = Mat :: eye（3，4，CV_32FC1） ; // 3x4矩阵，摄像机姿态
 float norm1 =（float）norm（H.col（0））; 
 float norm2 =（float）norm（H.col（1））; 
 float tnorm =（norm1 + norm2）/ 2.0f; //归一化值
 
 Mat p1 = H.col（0）; //指向H 
的第一列Mat p2 = pose.col（0）; //指向第一列姿势（空）
 
 cv :: normalize（p1，p2）; //归一化旋转，并复制列以构成
 
 p1 = H.col（1）; //指向H 
的第二列p2 = pose.col（1）; //指向第二列姿势（空）
 
 cv :: normalize（p1，p2）; //归一化旋转并复制列的姿势
 
 p1 = pose.col（0）; 
 p2 = pose.col（1）; 
 
 Mat p3 = p1.cross（p2）; //计算p1和p2的交叉乘积
 Mat c2 = pose.col（2）; //指向姿势的第三列
 p3.copyTo（c2）; //第三列是列一和二的交叉产品
 
 pose.col（3）= H.col（2）/ tnorm; // vector t [R | t]是姿势的最后一列
} 
  

方法工作形式我。祝你好运。

I have 4 coplanar points in a video (or image) representing a quad (not necessarily a square or rectangle) and I would like to be able to display a virtual cube on top of them where the corners of the cube stand exactly on the corners of the video quad.

Since the points are coplanar I can compute the homography between the corners of a unit square (i.e. [0,0] [0,1] [1,0] [1,1]) and the video coordinates of the quad.

From this homography I should be able to compute a correct camera pose, i.e. [R|t] where R is a 3x3 rotation matrix and t is a 3x1 translation vector so that the virtual cube lies on the video quad.

I have read many solutions (some of them on SO) and tried implementing them but they seem to work only in some "simple" cases (like when the video quad is a square) but do not work in most cases.

Here are the methods I tried (most of them are based on the same principles, only the computation of the translation are slightly different). Let K be the intrinsics matrix from the camera and H be the homography. We compute:

A = K-1 * H

Let a1,a2,a3 be the column vectors of A and r1,r2,r3 the column vectors of the rotation matrix R.

r1 = a1 / ||a1||
r2 = a2 / ||a2||
r3 = r1 x r2
t = a3 / sqrt(||a1||*||a2||)

The issue is that this does not work in most cases. In order to check my results, I compared R and t with those obtained by OpenCV's solvePnP method (using the following 3D points [0,0,0] [0,1,0] [1,0,0] [1,1,0]).

Since I display the cube in the same way, I noticed that in every case solvePnP provides correct results, while the pose obtained from the homography is mostly wrong.

In theory since my points are coplanar, it is possible to compute the pose from a homography but I couldn't find the correct way to compute the pose from H.

Any insights on what I am doing wrong?

Edit after trying @Jav_Rock's method

Hi Jav_Rock, thanks very much for your answer, I tried your approach (and many others as well) which seems to be more or less OK. Nevertheless I still happen to have some issues when computing the pose based on 4 coplanar point. In order to check the results I compare with results from solvePnP (which will be much better due to the iterative reprojection error minimization approach).

Here is an example:

Yellow cube: Solve PNP Black Cube: Jav_Rock's technique Cyan (and Purple) cube(s): some other techniques given the exact same results

As you can see, the black cube is more or less OK but doesn't seem well proportioned, although the vectors seem orthonormal.

EDIT2: I normalized v3 after computing it (in order to enforce orthonormality) and it seems to solve some problems as well.

解决方案

If you have your Homography, you can calculate the camera pose with something like this:

void cameraPoseFromHomography(const Mat& H, Mat& pose)
{
    pose = Mat::eye(3, 4, CV_32FC1);      // 3x4 matrix, the camera pose
    float norm1 = (float)norm(H.col(0));  
    float norm2 = (float)norm(H.col(1));  
    float tnorm = (norm1 + norm2) / 2.0f; // Normalization value

    Mat p1 = H.col(0);       // Pointer to first column of H
    Mat p2 = pose.col(0);    // Pointer to first column of pose (empty)

    cv::normalize(p1, p2);   // Normalize the rotation, and copies the column to pose

    p1 = H.col(1);           // Pointer to second column of H
    p2 = pose.col(1);        // Pointer to second column of pose (empty)

    cv::normalize(p1, p2);   // Normalize the rotation and copies the column to pose

    p1 = pose.col(0);
    p2 = pose.col(1);

    Mat p3 = p1.cross(p2);   // Computes the cross-product of p1 and p2
    Mat c2 = pose.col(2);    // Pointer to third column of pose
    p3.copyTo(c2);       // Third column is the crossproduct of columns one and two

    pose.col(3) = H.col(2) / tnorm;  //vector t [R|t] is the last column of pose
}

This method works form me. Good luck.

这篇关于基于4个共面点计算照相机姿态与单应性矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！