来自相应图像的相机运动 [英] Camera motion from corresponding images

问题描述

我正在尝试根据相应图像的运动来计算新的相机位置. 图像符合针孔相机模型.

I'm trying to calculate a new camera position based on the motion of corresponding images. the images conform to the pinhole camera model.

事实上，我没有得到有用的结果，所以我尝试描述我的程序，并希望有人能帮助我.

As a matter of fact, I don't get useful results, so I try to describe my procedure and hope that somebody can help me.

我使用SIFT匹配相应图像的特征，将它们与OpenCV的FlannBasedMatcher匹配，并使用OpenCV的findFundamentalMat(方法RANSAC)计算基本矩阵.

I match the features of the corresponding images with SIFT, match them with OpenCV's FlannBasedMatcher and calculate the fundamental matrix with OpenCV's findFundamentalMat (method RANSAC).

然后我通过相机固有矩阵(K)计算基本矩阵:

Then I calculate the essential matrix by the camera intrinsic matrix (K):

Mat E = K.t() * F * K;

我通过奇异值分解将基本矩阵分解为旋转和平移:

I decompose the essential matrix to rotation and translation with singular value decomposition:

SVD decomp = SVD(E);
Matx33d W(0,-1,0,
          1,0,0,
          0,0,1);
Matx33d Wt(0,1,0,
          -1,0,0,
           0,0,1);
R1 = decomp.u * Mat(W) * decomp.vt;
R2 = decomp.u * Mat(Wt) * decomp.vt;
t1 = decomp.u.col(2); //u3
t2 = -decomp.u.col(2); //u3

然后，我尝试通过三角剖分找到正确的解决方案. (这部分来自 http://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/，所以我认为应该可以正常工作.

Then I try to find the correct solution by triangulation. (this part is from http://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/ so I think that should work correct).

然后使用以下方法计算新职位:

The new position is then calculated with:

new_pos = old_pos + -R.t()*t;

其中new_pos& old_pos是向量(3x1)，R是旋转矩阵(3x3)，t是转换向量(3x1).

where new_pos & old_pos are vectors (3x1), R the rotation matrix (3x3) and t the translation vector (3x1).

不幸的是，我没有得到有用的结果，所以也许任何人都知道什么地方可能出了问题.

Unfortunately I got no useful results, so maybe anyone has an idea what could be wrong.

以下是一些结果(以防万一有人可以确认其中任何一个绝对错误):

Here are some results (just in case someone can confirm that any of them is definitely wrong):

F = [8.093827077399547e-07, 1.102681999632987e-06, -0.0007939604310854831;
     1.29246107737264e-06, 1.492629957878578e-06, -0.001211264339006535;
     -0.001052930954975217, -0.001278667878010564, 1]

K = [150, 0, 300;
    0, 150, 400;
    0, 0, 1]

E = [0.01821111092414898, 0.02481034499174221, -0.01651092283654529;
     0.02908037424088439, 0.03358417405226801, -0.03397110489649674;
     -0.04396975675562629, -0.05262169424538553, 0.04904210357279387]

t = [0.2970648246214448; 0.7352053067682792; 0.6092828956013705]

R = [0.2048034356172475, 0.4709818957303019, -0.858039396912323;
     -0.8690270040802598, -0.3158728880490416, -0.3808101689488421;
     -0.4503860776474556, 0.8236506374002566, 0.3446041331317597]

推荐答案

首先，您应该检查

x' * F * x = 0

用于您的点对应x'和x.当然，这仅适用于使用RANSAC进行基本矩阵估计的内部情况.

for your point correspondences x' and x. This should be of course only the case for the inliers of the fundamental matrix estimation with RANSAC.

此后，您必须像这样将点对应关系转换为归一化图像坐标(NCC)

Thereafter, you have to transform your point correspondences to normalized image coordinates (NCC) like this

xn = inv(K) * x
xn' = inv(K') * x'

其中，K'是第二个图像的固有相机矩阵，x'是第二个图像的点.我认为您的情况是K = K'.

where K' is the intrinsic camera matrix of the second image and x' are the points of the second image. I think in your case it is K = K'.

使用这些NCC，您可以像描述的那样分解基本矩阵.对三角化的相机坐标进行三角剖分，并检查三角剖分点的深度.但是要小心，在文献中他们说一个点足以获得正确的旋转和平移.根据我的经验，您应该检查一些要点，因为即使在RANSAC之后，也有可能是离群值.

With these NCCs you can decompose your essential matrix like you described. You triangulate the normalized camera coordinates and check the depth of your triangulated points. But be careful, in literature they say that one point is sufficient to get the correct rotation and translation. From my experience you should check a few points since one point can be an outlier even after RANSAC.

在分解基本矩阵之前，请确保E=U*diag(1,1,0)*Vt.对于投影矩阵的四个可能的选择，需要此条件才能获得正确的结果.

Before you decompose the essential matrix make sure that E=U*diag(1,1,0)*Vt. This condition is required to get correct results for the four possible choices of the projection matrix.

获得正确的旋转和平移后，您可以对所有点对应关系进行三角剖分(使用RANSAC进行基本矩阵估计的离群值).然后，您应该计算重新投影错误.首先，您要像这样计算重新投影的位置

When you've got the correct rotation and translation you can triangulate all your point correspondences (the inliers of the fundamental matrix estimation with RANSAC). Then, you should compute the reprojection error. Firstly, you compute the reprojected position like this

xp = K * P * X
xp' = K' * P' * X

其中，X是计算的(均匀)3D位置. P和P'是3x4投影矩阵.投影矩阵P通常由恒等式给出. P' = [R, t]由前3列和第3行中的旋​​转矩阵和第4列中的平移给出，因此P是3x4矩阵.仅当您将3D位置转换为同质坐标(即4x1向量而不是3x1)时，此方法才有效.然后，xp和xp'也是同构坐标，表示您的对应点的(重新投影的)二维位置.

where X is the computed (homogeneous) 3D position. P and P' are the 3x4 projection matrices. The projection matrix P is normally given by the identity. P' = [R, t] is given by the rotation matrix in the first 3 columns and rows and the translation in the fourth column, so that P is a 3x4 matrix. This only works if you transform your 3D position to homogeneous coordinates, i.e. 4x1 vectors instead of 3x1. Then, xp and xp' are also homogeneous coordinates representing your (reprojected) 2D positions of your corresponding points.

我认为

new_pos = old_pos + -R.t()*t;

是不正确的，因为首先，您仅平移old_pos，并且不旋转它，其次，您使用错误的向量平移了它.上面给出了正确的方法.

is incorrect since firstly, you only translate the old_pos and you do not rotate it and secondly, you translate it with a wrong vector. The correct way is given above.

因此，在计算了重新投影的点之后，您可以计算重新投影的误差.由于您正在使用均质坐标，因此必须对其进行归一化(xp = xp / xp(2)，除以最后一个坐标).这是由

So, after you computed the reprojected points you can calculate the reprojection error. Since you are working with homogeneous coordinates you have to normalize them (xp = xp / xp(2), divide by last coordinate). This is given by

error = (x(0)-xp(0))^2 + (x(1)-xp(1))^2

如果误差很大，例如10 ^ 2，则说明您的相机固有校准或旋转/平移不正确(也许两者均不正确).根据您的坐标系，您可以尝试反转投影矩阵.因此，您需要先将它们转换为齐次坐标，因为您无法反转3x4矩阵(没有伪逆).因此，添加第四行[0 0 0 1]，计算逆，然后删除第四行.

If the error is large such as 10^2 your intrinsic camera calibration or your rotation/translation are incorrect (perhaps both). Depending on your coordinate system you can try to inverse your projection matrices. On that account you need to transform them to homogeneous coordinates before since you cannot invert a 3x4 matrix (without the pseudo inverse). Thus, add the fourth row [0 0 0 1], compute the inverse and remove the fourth row.

还有另一件事带有重投影错误.通常，重投影误差是原始点对应关系(在每个图像中)与重投影位置之间的平方距离.您可以取平方根来获取两点之间的欧几里得距离.

There is one more thing with reprojection error. In general, the reprojection error is the squared distance between your original point correspondence (in each image) and the reprojected position. You can take the square root to get the Euclidean distance between both points.

这篇关于来自相应图像的相机运动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！