我如何处理基本矩阵? [英] What do I do with the fundamental matrix?

问题描述

我正在尝试从多个 2d 图像重建一个 3d 形状.我已经计算了一个基本矩阵，但现在我不知道该怎么做.

我在堆栈溢出和学术论文中发现了多个相互矛盾的答案.例如，.这也意味着，如果您有异常值或不正确的点对应关系，它会直接影响您的基本矩阵的质量.

另外，3幅图像之间的点对应关系也存在类似的结构，称为三焦张量.

仅使用基本矩阵的属性进行 3d 重建是不可能的，因为对极几何是两个视图之间的内在投影几何.它是独立于场景结构，仅取决于相机的内部参数和相对姿势."(HZ，第 239 页).

相机矩阵

参考您的问题如何从多个图像重建形状，您需要知道图像的相机矩阵(K'，K).相机矩阵是由相机焦距或主距(fx, fy)以及光学中心或主点(cx, cy)组成的3x3矩阵.

<小时>

您可以使用相机校准导出相机矩阵.

基本矩阵

当您知道相机矩阵时，您可以将基本矩阵扩展为基本矩阵 E.

<小时>

您可以很草率地说您的基本矩阵现在已校准".

基本矩阵可用于获得与第一张图像相比的第二张图像的旋转(旋转矩阵 R)和平移(向量 t)，仅达到投影重建.t 将是一个单位向量.为此，您可以使用 OpenCV 函数 decomposeEssentialMat 或 recoverPose(使用手性检查)或阅读 HZ 中的进一步详细说明.

投影矩阵

了解您的平移和旋转后，您可以为您的图像构建投影矩阵.投影矩阵定义为 .最后，您可以使用三角测量 (triangulatePoints) 来导出图像点的 3d 坐标.我建议使用后续的捆绑调整来获得正确的配置.openCV中也有sfm模块.

由于单应性或对极线知识对于 3d 重建不是必需的，因此我没有解释这些概念.

I am trying to reconstruct a 3d shape from multiple 2d images. I have calculated a fundamental matrix, but now I don't know what to do with it.

I am finding multiple conflicting answers on stack overflow and academic papers. For example, Here says you need to compute the rotation and translation matrices from the fundamental matrix.

Here says you need to find the camera matrices.

Here says you need to find the homographies.

Here says you need to find the epipolar lines.

Which is it?? (And how do I do it? I have read the H&Z book but I do not understand it. It says I can 'easily' use the 'direct formula' in result 9.14, but result 9.14 is neither easy nor direct to understand.)

Stack overflow wants code so here's what I have so far:

    # let's create some sample data

    Wpts = np.array([[1, 1, 1, 1],  # A Cube in world points
                     [1, 2, 1, 1],
                     [2, 1, 1, 1],
                     [2, 2, 1, 1],
                     [1, 1, 2, 1],
                     [1, 2, 2, 1],
                     [2, 1, 2, 1],
                     [2, 2, 2, 1]])


    Cpts = np.array([[0, 4, 0, 1],  #slightly up
                     [4, 0, 0, 1],
                     [-4, 0, 0, 1],
                     [0, -4, 0, 1]])
    Cangles = np.array([[0, -1, 0],  #slightly looking down
                        [-1, 0, 0],
                        [1, 0, 0],
                        [0,1,0]])



    views = []
    transforms = []
    clen = len(Cpts)
    for i in range(clen):
        cangle = Cangles[i]
        cpt = Cpts[i]

        transform = cameraTransformMatrix(cangle, cpt)
        transforms.append(transform)
        newpts = np.dot(Wpts, transform.T)
        view = cameraView(newpts)
        views.append(view)



H = cv2.findFundamentalMat(views[0], views[1])[0]
## now what???  How do I recover the cube shape?

Edit: I do not know the camera parameters

解决方案

Fundamental Matrix

At first, listen to the fundamental matrix song ;).

The Fundamental Matrix only shows the mathematical relationship between your point correspondences in 2 images (x' - image 2, x - image 1). "That means, for all pairs of corresponding points holds " (Wikipedia). This also means, that if you are having outlier or incorrect point correspondences, it directly affects the quality of your fundamental matrix.

Additionally, a similar structure exists for the relationship of point correspondences between 3 images which is called Trifocal Tensor.

A 3d reconstruction using exclusively the properties of the Fundamental Matrix is not possible because "The epipolar geometry is the intrinsic projective geometry between two views. It is independent of scene structure, and only depends on the cameras’ internal parameters and relative pose." (HZ, p.239).

Camera matrix

Refering to your question how to reconstruct the shape from multiple images you need to know the camera matrices of your images (K', K). The camera matrix is a 3x3 matrix composed of the camera focal lengths or principal distance (fx, fy) as well as the optical center or principal point (cx, cy).

---

You can derive your camera matrix using camera calibration.

Essential matrix

When you know your camera matrices you can extend your Fundamental Matrix to a Essential Matrix E.

---

You could say quite sloppy that your Fundamental Matrix is now "calibrated".

The Essential Matrix can be used to get the rotation (rotation matrix R) and translation (vector t) of your second image in comparison to your first image only up to a projective reconstruction. t will be a unit vector. For this purpose you can use the OpenCV functions decomposeEssentialMat or recoverPose (that uses the cheirality check) or read further detailed explanations in HZ.

Projection matrix

Knowing your translation and rotation you can build you projection matrices for your images. The projection matrix is defined as . Finally, you can use triangulation (triangulatePoints) to derive the 3d coordinates of your image points. I recommend using a subsequent bundle adjustment to receive a proper configuration. There is also a sfm module in openCV.

Since homography or epipolar line knowledge is not essentially necessary for the 3d reconstruction I did not explain these concepts.

这篇关于我如何处理基本矩阵?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！