我如何处理基本矩阵? [英] What do I do with the fundamental matrix?

问题描述

我正在尝试从多个2d图像中重建3d形状. 我已经计算了一个基本矩阵，但是现在我不知道该怎么处理.

I am trying to reconstruct a 3d shape from multiple 2d images. I have calculated a fundamental matrix, but now I don't know what to do with it.

我在堆栈溢出和学术论文中发现了多个相互矛盾的答案. 例如，此处表示您需要计算旋转和平移矩阵从基本矩阵开始.

I am finding multiple conflicting answers on stack overflow and academic papers. For example, Here says you need to compute the rotation and translation matrices from the fundamental matrix.

此处说，您需要找到相机矩阵.

Here says you need to find the camera matrices.

此处说，您需要找到单应字.

Here says you need to find the homographies.

此处说您需要找到对极线.

Here says you need to find the epipolar lines.

是哪个? (我该怎么办?我已阅读

Which is it?? (And how do I do it? I have read the H&Z book but I do not understand it. It says I can 'easily' use the 'direct formula' in result 9.14, but result 9.14 is neither easy nor direct to understand.)

堆栈溢出需要代码，所以这是我到目前为止所拥有的:

Stack overflow wants code so here's what I have so far:

    # let's create some sample data

    Wpts = np.array([[1, 1, 1, 1],  # A Cube in world points
                     [1, 2, 1, 1],
                     [2, 1, 1, 1],
                     [2, 2, 1, 1],
                     [1, 1, 2, 1],
                     [1, 2, 2, 1],
                     [2, 1, 2, 1],
                     [2, 2, 2, 1]])


    Cpts = np.array([[0, 4, 0, 1],  #slightly up
                     [4, 0, 0, 1],
                     [-4, 0, 0, 1],
                     [0, -4, 0, 1]])
    Cangles = np.array([[0, -1, 0],  #slightly looking down
                        [-1, 0, 0],
                        [1, 0, 0],
                        [0,1,0]])



    views = []
    transforms = []
    clen = len(Cpts)
    for i in range(clen):
        cangle = Cangles[i]
        cpt = Cpts[i]

        transform = cameraTransformMatrix(cangle, cpt)
        transforms.append(transform)
        newpts = np.dot(Wpts, transform.T)
        view = cameraView(newpts)
        views.append(view)



H = cv2.findFundamentalMat(views[0], views[1])[0]
## now what???  How do I recover the cube shape?

我不知道相机参数

推荐答案

基本矩阵

首先，请听基本矩阵歌曲;).

仅基本矩阵 在2张图像(x'-图像2，x-图像1)中显示了点对应关系之间的数学关系. 这意味着，对于所有成对的对应点，其"(维基百科).这也意味着，如果您有异常值或不正确的点对应关系，则将直接影响基本矩阵的质量.

The Fundamental Matrix only shows the mathematical relationship between your point correspondences in 2 images (x' - image 2, x - image 1). "That means, for all pairs of corresponding points holds " (Wikipedia). This also means, that if you are having outlier or incorrect point correspondences, it directly affects the quality of your fundamental matrix.

另外， 3张图像之间的点对应关系也存在类似的结构，称为三焦点张量.

Additionally, a similar structure exists for the relationship of point correspondences between 3 images which is called Trifocal Tensor.

不可能完全使用基本矩阵的属性进行3d重建，因为对极几何是两个视图之间的固有投影几何. 与场景结构无关，仅取决于相机的内部参数 和相对姿势."(HZ，第239页).

A 3d reconstruction using exclusively the properties of the Fundamental Matrix is not possible because "The epipolar geometry is the intrinsic projective geometry between two views. It is independent of scene structure, and only depends on the cameras’ internal parameters and relative pose." (HZ, p.239).

关于如何从多个图像中重建形状的问题，您需要了解图像的相机矩阵(K'，K).摄影机矩阵是一个3x3矩阵，由摄影机焦距或主要距离(fx，fy)以及光学中心或主要点(cx，cy)组成.

Refering to your question how to reconstruct the shape from multiple images you need to know the camera matrices of your images (K', K). The camera matrix is a 3x3 matrix composed of the camera focal lengths or principal distance (fx, fy) as well as the optical center or principal point (cx, cy).

您可以使用相机校准来导出相机矩阵.

You can derive your camera matrix using camera calibration.

了解相机矩阵后，可以将基本矩阵扩展为基本矩阵E.

When you know your camera matrices you can extend your Fundamental Matrix to a Essential Matrix E.

您可能会很草率地说您的基本面现在已经校准"了.

You could say quite sloppy that your Fundamental Matrix is now "calibrated".

仅在投影重建之前，基本矩阵可用于获取第二张图片与第一张图片相比的旋转(旋转矩阵R)和平移(矢量t). t将是一个单位向量.为此，您可以使用OpenCV函数decomposeEssentialMat或recoverPose(使用检查性)或阅读HZ中的更多详细说明.

The Essential Matrix can be used to get the rotation (rotation matrix R) and translation (vector t) of your second image in comparison to your first image only up to a projective reconstruction. t will be a unit vector. For this purpose you can use the OpenCV functions decomposeEssentialMat or recoverPose (that uses the cheirality check) or read further detailed explanations in HZ.

了解平移和旋转后，您可以为图像建立投影矩阵.投影矩阵定义为.最后，您可以使用三角剖分(triangulatePoints)得出图像点的3d坐标.我建议使用后续的包调整来接收正确的配置. openCV中还有一个sfm模块.

Knowing your translation and rotation you can build you projection matrices for your images. The projection matrix is defined as . Finally, you can use triangulation (triangulatePoints) to derive the 3d coordinates of your image points. I recommend using a subsequent bundle adjustment to receive a proper configuration. There is also a sfm module in openCV.

由于3D重建本质上不需要单应性或对极线知识，所以我没有解释这些概念.

Since homography or epipolar line knowledge is not essentially necessary for the 3d reconstruction I did not explain these concepts.

这篇关于我如何处理基本矩阵?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！