在校准的立体视觉装备中，如何获得“相机矩阵"?需要实现 3D 三角测量算法? [英] In a calibrated stereo-vision rig, how does one obtain the "camera matrices" needed for implementing a 3D triangulation algorithm?

问题描述

我正在尝试实现 Hartley & 的(相对简单的)线性齐次 (DLT) 3D 三角测量方法Zisserman 的多视图几何"(第 12.2 节)，目的是在未来实现其完整的最佳算法".现在，基于 this question，我正试图让它在 Matlab 中工作，稍后会将其移植到 C++ 和 OpenCV 中，并在此过程中进行一致性测试.

I am trying to implement the (relatively simple) linear homogeneous (DLT) 3D triangulation method from Hartley & Zisserman's "Multiple View Geometry" (sec 12.2), with the aim of implementing their full, "optimal algorithm" in the future. Right now, based on this question, I'm trying to get it to work in Matlab, and will later port it into C++ and OpenCV, testing for conformity along the way.

问题是我不确定如何使用我拥有的数据.我已经校准了我的立体装置，并获得了两个固有的相机矩阵、两个畸变系数向量、两个相机相关的旋转矩阵和平移向量，以及基本矩阵和基本矩阵.我还有两个点的 2D 坐标，它们应该是两个图像坐标系中单个 3D 点的对应关系(分别由第一个和第二个相机拍摄).

The problem is that I'm unsure how to use the data I have. I have calibrated my stereo rig, and obtained the two intrinsic camera matrices, two vectors of distortion coefficients, the rotation matrix and translation vector relating the two cameras, as well as the essential and fundamental matrices. I also have the 2D coordinates of two points that are supposed to be correspondences of a single 3D point in the coordinate systems of the two images (taken by the 1st and 2nd camera respectively).

该算法将两个点坐标和两个 4x3相机矩阵"P 和 P' 作为输入.这些显然不是从校准中获得的内在相机矩阵(M，M')，因为它们是 3x3，而且因为单独使用它们的投影将 3D 点放在两个不同的坐标系中，即 - 外在的(旋转/平移)数据丢失.

The algorithm takes as input the two point coordinates and two 4x3 "camera matrices" P and P'. These aren't obviously the intrinsic camera matrices (M, M') obtained from the calibration, because for one they are 3x3, and also because projection using them alone puts a 3D point in two distinct coordinate systems, that is - the extrinsic (rotation/translation) data is missing.

H&Z 书包含有关使用 SVD 分解从基本矩阵或基本矩阵中恢复所需矩阵的信息(第 9 章)，但它本身也存在其他问题(例如尺度模糊).我觉得我不需要那个，因为我已经明确定义了旋转和平移.

The H&Z book contains information (chapter 9) on recovering the required matrices from either the fundamental or the essential matrix using SVD decomposition, but with additional problems of its own (e.g. scale ambiguity). I feel I don't need that, since I have the rotation and translation explicitly defined.

然后的问题是:使用第一个内在矩阵是否正确，将额外的零列作为第一个相机矩阵"(P = [M | 0])，然后将第二个内在矩阵乘以由旋转矩阵和平移向量组成的外在矩阵作为额外列，以获得第二个所需的相机矩阵"(P'=M'*[R|t])?还是应该换一种方式?

The question then is: would it be correct to use the first intrinsic matrix, with an extra column of zeros as the first "camera matrix" (P=[M|0]), and then multiply the second intrinsic matrix by a extrinsic matrix composed from the rotation matrix and the translation vector as an extra column to obtain the second required "camera matrix" (P'=M'*[R|t])? Or should it be done differently?

谢谢！

推荐答案

我手头没有 H&Z - 但是他们关于这个主题的旧 CVPR 教程是 这里(其他人可以看看这个问题).

I don't have my H&Z to hand - but their old CVPR tutorial on the subject is here (for anyone else to have a look at w.r.t this question).

为了清楚起见(并使用他们的术语)，投影矩阵 P 从欧几里得 3 空间点 (X) 映射到图像点 (x) 为:

Just for clarity (and to use their terminology) the projection matrix P maps from Euclidean 3-space point (X) to an image point (x) as:

x = PX

地点:

P = K[ R | t ]

由 (3x3) 相机校准矩阵 K 和 (3x3) 旋转矩阵 R 和平移向量 (3x1) t 定义.

defined by the (3x3) camera calibration matrix K and the (3x3) rotation matrix R and translation vector (3x1) t.

问题的关键似乎是如何使用你的两个相机P和P'进行三角测量.

The crux of the matter seems to be how to then perform triangulation using your two cameras P and P'.

我相信你是在提议世界原点位于第一个相机P，因此:

I believe you are proposing that the world origin is located at a the first camera P, thus:

P = K [ I | 0]

和

P' = K' [ R | t ]

然后我们在基本矩阵 F 中寻求重建，使得:

What we then seek for reconstruction in the Fundamental Matrix F such that:

x' F x = 0

矩阵 F 当然可以通过多种方式计算(有时更常见于未校准的图像！)但在这里我认为您可能希望根据已经校准的相机矩阵来计算以上为:

The matrix F can of course be computed any number of ways (sometimes more commonly from uncalibrated images!) but here I think you might want to do it on the basis of your already calibrated camera matrices above as:

F = [P' C]_x P' pinv(P)

其中 C = (0 1) 是第一个相机的中心，pinv(P) 是 P 的伪逆._x 表示文献中用于矩阵乘法计算向量积的符号.

Where C = (0 1) is the centre of first camera and pinv(P) is the pseudo-inverse of P. The _x indicates the notation used in the literature for matrix multiplication to calculate the vector product.

然后您可以执行基本矩阵 F 的因式分解(通过 SVD 或直接方法执行).

You can then perform a factorization of the fundamental matrix F (performed via SVD or direct method).

F = [t]_x M

因此，正如您所说的那样，我们可以根据以下条件直接计算三角测量:

And hence, as you correctly state, we can then compute triangulation directly based on:

P = [ I | 0 ] 

和

P' = [ M | t ]

使用这些来执行三角测量应该相对简单(假设校准良好、没有噪音等)

Using these to perform triangulation should then be relatively straightforward (assuming good calibration, lack of noise, etc. etc.)

这篇关于在校准的立体视觉装备中，如何获得“相机矩阵"?需要实现 3D 三角测量算法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！