基于 4 个共面点的单应矩阵计算相机姿态 [英] Computing camera pose with homography matrix based on 4 coplanar points

问题描述

我在代表四边形(不一定是正方形或矩形)的视频(或图像)中有 4 个共面点，我希望能够在它们的顶部显示一个虚拟立方体，立方体的角正好站立在视频四边形的角落.

I have 4 coplanar points in a video (or image) representing a quad (not necessarily a square or rectangle) and I would like to be able to display a virtual cube on top of them where the corners of the cube stand exactly on the corners of the video quad.

由于这些点是共面的，我可以计算单位正方形的角(即 [0,0] [0,1] [1,0] [1,1])和四边形的视频坐标之间的单应性.

Since the points are coplanar I can compute the homography between the corners of a unit square (i.e. [0,0] [0,1] [1,0] [1,1]) and the video coordinates of the quad.

根据这个单应性，我应该能够计算出正确的相机姿势，即 [R|t]，其中 R 是 3x3 旋转矩阵，t 是 3x1 平移向量，因此虚拟立方体位于视频四边形上.

From this homography I should be able to compute a correct camera pose, i.e. [R|t] where R is a 3x3 rotation matrix and t is a 3x1 translation vector so that the virtual cube lies on the video quad.

我已经阅读了许多解决方案(其中一些在 SO 上)并尝试实施它们，但它们似乎仅在某些简单"情况下有效(例如视频四边形是正方形时)，但在大多数情况下不起作用.

I have read many solutions (some of them on SO) and tried implementing them but they seem to work only in some "simple" cases (like when the video quad is a square) but do not work in most cases.

以下是我尝试的方法(大部分都是基于相同的原理，只是翻译的计算略有不同).设 K 是来自相机的内在矩阵，H 是单应性矩阵.我们计算:

Here are the methods I tried (most of them are based on the same principles, only the computation of the translation are slightly different). Let K be the intrinsics matrix from the camera and H be the homography. We compute:

A = K-1 * H

设 a1,a2,a3 为 A 的列向量，r1,r2,r3 为旋转矩阵 R 的列向量.

Let a1,a2,a3 be the column vectors of A and r1,r2,r3 the column vectors of the rotation matrix R.

r1 = a1 / ||a1||
r2 = a2 / ||a2||
r3 = r1 x r2
t = a3 / sqrt(||a1||*||a2||)

问题是这在大多数情况下不起作用.为了检查我的结果，我将 R 和 t 与通过 OpenCV 的 solvePnP 方法获得的结果进行了比较(使用以下 3D 点 [0,0,0] [0,1,0] [1,0,0] [1,1,0]).

The issue is that this does not work in most cases. In order to check my results, I compared R and t with those obtained by OpenCV's solvePnP method (using the following 3D points [0,0,0] [0,1,0] [1,0,0] [1,1,0]).

由于我以相同的方式显示立方体，我注意到在每种情况下，solvePnP 都能提供正确的结果，而从单应性获得的姿势大多是错误的.

Since I display the cube in the same way, I noticed that in every case solvePnP provides correct results, while the pose obtained from the homography is mostly wrong.

理论上，由于我的点是共面的，可以从单应性计算姿势，但我找不到从 H 计算姿势的正确方法.

In theory since my points are coplanar, it is possible to compute the pose from a homography but I couldn't find the correct way to compute the pose from H.

对我做错了什么有任何见解吗?

Any insights on what I am doing wrong?

尝试@Jav_Rock 的方法后编辑

您好 Jav_Rock，非常感谢您的回答，我尝试了您的方法(以及许多其他方法)，这似乎或多或少是可以的.尽管如此，在计算基于 4 个共面点的姿势时，我仍然碰巧遇到一些问题.为了检查结果，我与来自 solvePnP 的结果进行了比较(由于迭代重投影误差最小化方法，结果会好得多).

Hi Jav_Rock, thanks very much for your answer, I tried your approach (and many others as well) which seems to be more or less OK. Nevertheless I still happen to have some issues when computing the pose based on 4 coplanar point. In order to check the results I compare with results from solvePnP (which will be much better due to the iterative reprojection error minimization approach).

这是一个例子:

• 黄色立方体:解决 PNP
• Black Cube:Jav_Rock 的技术
• 青色(和紫色)立方体:一些其他技术给出了完全相同的结果

如您所见，黑色立方体或多或少还可以，但看起来比例不太好，尽管矢量看起来是正交的.

As you can see, the black cube is more or less OK but doesn't seem well proportioned, although the vectors seem orthonormal.

我在计算 v3 后对其进行了标准化(为了强制正交性)，它似乎也解决了一些问题.

I normalized v3 after computing it (in order to enforce orthonormality) and it seems to solve some problems as well.

推荐答案

如果你有 Homography，你可以用这样的东西来计算相机姿势:

If you have your Homography, you can calculate the camera pose with something like this:

void cameraPoseFromHomography(const Mat& H, Mat& pose)
{
    pose = Mat::eye(3, 4, CV_32FC1);      // 3x4 matrix, the camera pose
    float norm1 = (float)norm(H.col(0));  
    float norm2 = (float)norm(H.col(1));  
    float tnorm = (norm1 + norm2) / 2.0f; // Normalization value

    Mat p1 = H.col(0);       // Pointer to first column of H
    Mat p2 = pose.col(0);    // Pointer to first column of pose (empty)

    cv::normalize(p1, p2);   // Normalize the rotation, and copies the column to pose

    p1 = H.col(1);           // Pointer to second column of H
    p2 = pose.col(1);        // Pointer to second column of pose (empty)

    cv::normalize(p1, p2);   // Normalize the rotation and copies the column to pose

    p1 = pose.col(0);
    p2 = pose.col(1);

    Mat p3 = p1.cross(p2);   // Computes the cross-product of p1 and p2
    Mat c2 = pose.col(2);    // Pointer to third column of pose
    p3.copyTo(c2);       // Third column is the crossproduct of columns one and two

    pose.col(3) = H.col(2) / tnorm;  //vector t [R|t] is the last column of pose
}

这种方法对我有用.祝你好运.

This method works form me. Good luck.

这篇关于基于 4 个共面点的单应矩阵计算相机姿态的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！