基本矩阵的分解：验证R和T的四种可能解决方案 [英] Decomposition of Essential Matrix: Validation of the four possible solutions for R and T
问题描述
我想使用OpenCV做一些运动结构。
到目前为止，我有基本的矩阵和基本矩阵。我的问题是我有2个可能的解决方案为R和2可能的解决方案的T，导致总体姿势的4个解决方案，其中只有4个解决方案中的一个是正确的。
如何找到正确的解决方案？
这是我的代码：
private void calculateRT（Mat E，Mat R，Mat T）{
Mat w = new Mat（）;
Mat u = new Mat（）;
Mat vt = new Mat（）;
Mat diag = new Mat（3,3，CvType.CV_64FC1）;
double [] diagVal = {1,0,0,0,1,0,0,0,1};
diag.put（0，0，diagVal）;
Mat newE = new Mat（3,3，CvType.CV_64FC1）;
Core.SVDecomp（E，w，u，vt，Core.DECOMP_SVD）;
Core.gemm（u，diag，1，vt，1，newE）;
Core.SVDecomp（newE，w，u，vt，Core.DECOMP_SVD）;
publishProgress（U：+ u.dump（））;
publishProgress（W：+ w.dump（））;
publishProgress（vt：+ vt.dump（））;
double [] W_Values = {0，-1,0,1,0,0,0,0,1};
Mat W = new Mat（new Size（3,3），CvType.CV_64FC1）;
W.put（0，0，W_Values）;
double [] Wt_values = {0,1,0-1,0,0,0,0,1};
Mat Wt = new Mat（new Size（3,3），CvType.CV_64FC1）;
Wt.put（0,0，Wt_values）;
Mat R1 = new Mat（）;
Mat R2 = new Mat（）;
// u * W * vt = R
Core.gemm（u，Wt，1，vt，1，R2）
Core.gemm（u，W，1，vt，1，R1）;
publishProgress（R：+ R.dump（））;
// + - T（T的2个可能解）
Mat T1 = new Mat（）;
Mat T2 = new Mat（）;
// T = u.t
u.col（2）.copyTo（T1）;
publishProgress（T：+ T.dump（））;
Core.multiply（T，new Scalar（-1.0，-1.0，-1.0），T2）;
// TODO这里我必须为R1 R2和T1找到正确的组合T2
}
当从它们的基本矩阵重建两个相机的相对欧几里德姿态时，存在理论模糊性。这种模糊性与以下事实相关：给定图像中的2D点，经典针孔相机模型不能判断相应的3D点是在相机前面还是在相机后面。为了消除这种不确定性，您需要知道图像中的一个点对应：因为这两个2D点被假定为位于两个相机前面的单个3D点的投影（因为它在两个图像中都是可见的）这将允许选择正确的R和T.
为此，一个方法在下面博士论文的第6.1.4节（p47）中解释：几何，约束和三重张量的计算，由C.Ressl（ PDF ）。下面给出了这种方法的概要。我将用x1和x2表示两个相应的2D点，用K1和K2表示两个相机矩阵，用E12表示基本矩阵。
i。计算基本矩阵 E12 = U * S * V'的SVD。如果 det（U） 0 设置 U = -U 。如果 det（V） 0 设置 V = -V 。
定义 W = [0，-1,0; 1,0,0; 0,0,1] ， R2 = U * W * V'和 code>
iii。定义 M = [R2'* T2] x ， X1 = M * inv（K1）* x1 c $ c> X2 = M * R2'* inv（K2）* x2
<如果
X1（3）* X2（3） 0 ，设置 R2 = U * W'* V'并重新计算 M code> X1 v。如果 X1（3） 0 设置 T2 = -T2
vi。定义 P1_E = K1 * [I | 0] 和 P2_E = K2 * [R2 | T2]
符号'表示转置和符号 [。] x 。对应于偏斜对称运算符。在3x1向量上应用倾斜对称运算符 e = [e_1; e_2; e_3] 会产生以下结果（请参阅维基百科中关于跨产品）：
[e] x = [0，-e_3，e_2; e_3,0，-e_1; -e_2，e_1,0]
最后，请注意 T2 将始终为1，因为它是正交矩阵的列之一。这意味着您将无法恢复两个摄像机之间的真实距离。为此，您需要知道场景中两个点之间的真实距离，并将其考虑在内以计算相机之间的真实距离。
I want to do some Structure-from-Motion using OpenCV. So far I have the fundamentalmatix and the essentialmatrix. Having the essentialmatrix I am doing SVD for getting R and T.
My problem is that I have 2 possible solutions for R and 2 possible solutions for T which leads to 4 solutions for the overall pose, where only one of the 4 solutions is the right one. How can I find the correct solution?
Here is my code:
private void calculateRT(Mat E, Mat R, Mat T){
Mat w = new Mat();
Mat u = new Mat();
Mat vt = new Mat();
Mat diag = new Mat(3,3,CvType.CV_64FC1);
double[] diagVal = {1,0,0,0,1,0,0,0,1};
diag.put(0, 0, diagVal);
Mat newE = new Mat(3,3,CvType.CV_64FC1);
Core.SVDecomp(E, w, u, vt, Core.DECOMP_SVD);
Core.gemm(u, diag, 1, vt, 1, newE);
Core.SVDecomp(newE, w, u, vt, Core.DECOMP_SVD);
publishProgress("U: " + u.dump());
publishProgress("W: " + w.dump());
publishProgress("vt:" + vt.dump());
double[] W_Values = {0,-1,0,1,0,0,0,0,1};
Mat W = new Mat(new Size(3,3), CvType.CV_64FC1);
W.put(0, 0, W_Values);
double[] Wt_values = {0,1,0-1,0,0,0,0,1};
Mat Wt = new Mat(new Size(3,3), CvType.CV_64FC1);
Wt.put(0,0,Wt_values);
Mat R1 = new Mat();
Mat R2 = new Mat();
// u * W * vt = R
Core.gemm(u, Wt, 1, vt, 1, R2);
Core.gemm(u, W, 1, vt, 1, R1);
publishProgress("R: " + R.dump());
// +- T (2 possible solutions for T)
Mat T1 = new Mat();
Mat T2 = new Mat();
// T = u.t
u.col(2).copyTo(T1);
publishProgress("T : " + T.dump());
Core.multiply(T, new Scalar(-1.0, -1.0, -1.0), T2);
// TODO Here I have to find the correct combination for R1 R2 and T1 T2
}
There is a theoretical ambiguity when reconstructing the relative euclidian poses of two cameras from their fundamental matrix. This ambiguity is linked to the fact that, given a 2D point in an image, the classic pinhole camera model cannot tell whether the corresponding 3D point is in front of the camera or behind the camera. In order to remove this ambiguity, you need to know one point correspondence in the images: as these two 2D points are assumed to be the projections of a single 3D point lying in front of both cameras (since it is visible in both images), this will enable choosing the right R and T.
For that purpose, one method is explained in § 6.1.4 (p47) of the following PhD thesis: "Geometry, constraints and computation of the trifocal tensor", by C.Ressl (PDF). The following gives the outline of this method. I'll denote the two corresponding 2D points by x1 and x2, the two camera matrices by K1 and K2 and the essential matrix by E12.
i. Compute the SVD of the essential matrix E12 = U * S * V'. If det(U) < 0 set U = -U. If det(V) < 0 set V = -V.
ii. Define W = [0,-1,0; 1,0,0; 0,0,1], R2 = U * W * V' and T2 = third column of U
iii. Define M = [ R2'*T2 ]x, X1 = M * inv(K1) * x1 and X2 = M * R2' * inv(K2) * x2
iv. If X1(3) * X2(3) < 0, set R2 = U * W' * V' and recompute M and X1
v. If X1(3) < 0 set T2 = -T2
vi. Define P1_E = K1 * [ I | 0 ] and P2_E = K2 * [ R2 | T2 ]
The notation ' denotes the transpose and the notation [.]x used in step iii. corresponds to the skew-symetric operator. Applying the skew-symmetric operator on a 3x1 vector e = [e_1; e_2; e_3] results in the following (see the Wikipedia article on cross-product):
[e]x = [0,-e_3,e_2; e_3,0,-e_1; -e_2,e_1,0]
Finally, note that the norm of T2 will always be 1, since it is one of the column of an orthogonal matrix. This means that you won't be able to recover the true distance between the two cameras. For that purpose, you need to know the true distance between two points in the scene and take that into account to calculate the true distance between the cameras.
这篇关于基本矩阵的分解：验证R和T的四种可能解决方案的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
