从运动结构，给定 2D 图像点对应关系重建 3D 点云 [英] Structure from Motion, Reconstruct the 3D Point Cloud given 2D Image points correspondence

问题描述

用例

• 一个物体围绕它的中心以不同的速度旋转
• 固定摄像头正在观察物体
• 给定 2D 图像点对应关系重建 3D 点云
• 当物体旋转时，相机会看到它的不同部分，因此，不同的点和;检测到对应关系.

场景

  a.N 图片
  b.N-1 图像对
  c.N-1个二维点对应(两个二维点数组)

  a. N Images
  b. N-1 Image pairs
  c. N-1 2D Point correspondence ( Two 2D Points arrays )

实施

对于每一个(N-1)个二维点对应

For each of the (N-1) 2D Points correspondences

1. 计算相机相对姿势
2. 三角测量得到 3D 点
3. 对于每个 2 3D 点数组，使用 [c] 中给出的 2D 对应关系导出对应关系
4. 使用 3D 对应导出@[3] 导出每个对象 3D 点的轨迹，从而为每个对象点/顶点生成单个轨迹

结果:

A (N–2) 个 3D 点阵列、对应关系、相机姿势和轨迹(每个对象点一个轨迹)

A (N–2) 3D Point arrays, Correspondences, camera poses and Tracks ( one track for each object point )

考虑的解决问题的方法:

鉴于三角测量结果精确到一个比例，计算点云.
  A.每个三角测量结果和相机相对平移
    以非齐次坐标表示(每个结果都有不同的比例).
  B.假设对象结构是固体，因此不会改变，
    对于所有相机姿势，每个 3D 点到其中心的距离应该相同.
  C.考虑到 [B]，[A] 处的所有三角化 3D 点和相机平移
      可以转换为齐次坐标系.
  D.选择一个相机姿势并将每个轨迹中的第一个点(定义 @ [4] )
     转换为该相机姿势(通过累积相机的倒数变换
     Pose )，结果，预期的点可以.

Given that the triangulation result is accurate up to a scale, calculate the point cloud.
  A. Each of the triangulation results and Camera relative Translations are
      expressed in NON-homogeneous coordinates ( each result has a different scale ).
  B. Under the assumption that the object structure is solid and thus, does not change,
      the distance of each of the 3D points to its center should be identical for all camera poses.
  C. Having [B] in mind, all triangulated 3D points at [A] and Cameras Translations
      can be converted to a homogeneous coordinate system.
  D. Select one of the Camera poses and Transform the first Point in each Track ( defined @ [4] )
      to that Camera Pose ( Transform by the inverse of the accumulated Camera
      Pose ), resulting, the expected point could.

推荐答案

重构一个对象是正确的过程.去年我在我们大学的一个项目中研究了这个主题.我的经验是用手移动相机重建一个物体并不容易.

It is the right procedure to reconstructe an object. I worked on this topic the last year at a project at our University. The experience I made is that it isn't easy to reconstruct a object by hand moving camera.

匹配

首先要考虑兴趣点的匹配.对于这点，SURF 和 SIFT 是很好的匹配方法.当物体移动小于 15° 时，您可以考虑使用比普通 SURF 快一点的 USURF(有关更多详细信息，请查看 SURF 论文).在我们的项目中，我们决定在 OpenCV 中使用光流，它看起来有点慢，但对异常值更加稳健.您的对象只是在旋转，因此您也可以考虑使用它.

First you have to think about the matching of intereset points. SURF and SIFT are good matching methods for this points. When the object is moving less then 15° you can think about to use USURF which is a bit faster then the normal SURF (for more details watch at the SURF paper). In our project we decided to us Optical Flow in OpenCV it looks a bit slower but was more robust about outliers. Your object is only rotating so you can think to use this, too.

矩阵评估

接下来是评估新相机矩阵的结果.你有没有可能找出物体旋转了多少(比如一些步进电机或其他东西)?因此，您可以将计算结果与电机的步数进行比较.如果它高于阈值，您就知道计算很糟糕.但是请注意，有些步进电机的精度不是很好，但可以通过一些实验来提供更多信息.

Next is evaluating your results of the new camera matrix. Do you have a possibility to find out how much the object was rotated (like some step motor or something)? So you can compare your computed results with the steps of the motor. If it is higher then threshold you know the computation was bad. But be carfull the precision of some step motors is not so good, but some experiments could bring more informations about that.

云评估

有一些很好的方法来评估计算出的云.最简单的方法是计算云的重投影误差.为此，您只需反转重建并查看计算图像与原始对应点的距离.另一个测试是检查所有点是否都在相机的前面.通过计算，这些点可能位于相机的前后.我知道当两个摄像头彼此关闭时会发生这种情况，并且三角测量也会终止.

There are some nice ways to evaluate the computed cloud. The easiest way is to compute the reprojection error of the cloud. For that you just reverse your reconstruction and look how far the computed images points away from the original corresponding points. A other Test is to check if all points are infronte of the camera. By computing it can happend that the points lie infront and behind the camera. I understand it can happend when both camera are to close each other, and the triangulation terminates as well.

第一对图像

我不确定静态相机是否需要执行此步骤.但首先我们必须计算一个基本矩阵.我们尝试使用匹配最多的图像对来提取它们，并使用 RANSAC 版本获得最佳结果.但是也许您可以尝试放置对象，使其在第一次拍摄时在前面具有最多的兴趣点.

Iam not sure if this step is necessary with a static camera. But first of all we had to calculate a Fundamental matrix. We made the experience to use the Image pair that has the most matches to extract them and use the RANSAC version give the best results. But maybe you can try to place the object so that it has the most Intereset points in the front for the first shot.

关注图像对

真正有效的是从现有的点云中提取新的相机位置，该点云之前是从旧图像对计算出来的.为此，您还记得之前图像的 2D 3D 对应关系.它称为 PerspectivenPoint 相机姿势估计 (PnP).

What worked really well is to extract the new camera positions from the exisiting point cloud which was computed from old image pairs before. For that you have remember the 2D 3D correspondenc of images before. It is called Perspective­n­Point Camera Pose Estimation (PnP).

最后我们有一些好的和坏的结果.这取决于扫描对象.以下是一些对我有帮助的论文:

At the end we had some good and bad results. It was depending on the scanning object. Here are some papers which helped me:

模拟世界

实时度量 3D 重建

这篇关于从运动结构，给定 2D 图像点对应关系重建 3D 点云的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！