通过点对应检测/校正照片变形 [英] Detecting/correcting Photo Warping via Point Correspondences

问题描述

我意识到有很多罐蠕虫与我要问的有关，但我必须从某个地方开始.基本上，我要问的是:

I realize there are many cans of worms related to what I'm asking, but I have to start somewhere. Basically, what I'm asking is:

给出一个用未知相机拍摄的场景的两张照片，我能在多大程度上确定这些照片之间的(相对)翘曲?

下面是1904年世界博​​览会的两幅图像.它们是在无线电报塔上的不同高度拍摄的，因此摄像机或多或少在垂直方向上排成一列.我的目标是根据这些照片和其他照片创建区域模型(如果需要的话，在Blender中).我不需要的是全自动解决方案，例如，手动选取点和特征没有问题.

Below are two images of the 1904 World's Fair. They were taken at different levels on the wireless telegraph tower, so the cameras are more or less vertically in line. My goal is to create a model of the area (in Blender, if it matters) from these and other photos. I'm not looking for a fully automated solution, e.g., I have no problem with manually picking points and features.

在过去的一个月中，我自学了有关投射变换和对极几何的知识.对于一些照片，我可以通过从点对应关系中找到基本矩阵 F 来做得很好.但是下面的两个正在给我带来麻烦.我怀疑是否存在某种变形-可能只是宽高比的变化，也许还不止于此.

Over the past month, I've taught myself what I can about projective transformations and epipolar geometry. For some pairs of photos, I can do pretty well by finding the fundamental matrix F from point correspondences. But the two below are causing me problems. I suspect that there's some sort of warping - maybe just an aspect ratio change, maybe more than that.

我的过程如下:

1. 我发现两张照片之间的对应关系(如下所示的红色锯齿线).
2. 我通过Matlab(实际上是八度)运行点对以找到子极.目前，我正在使用Peter Kovesi的 彼得的计算机视觉功能.
3. 在Blender中，我设置了两个摄像头，图像重叠.我根据消失点确定了第一台相机的方向.我还将根据消失点确定焦距.我使用子极和两个点对中的一个(下面的位置在演奏台的顶部)将第二个照相机相对于第一个照相机定向.
4. 对于每个对点，我从每个摄像机投射一条光线通过其采样点，并标记该对点的最接近覆盖度(下面的浅黄色). 我意识到这会遗漏基本矩阵中的信息-参见下文.

1. I find correspondences between the two photos (the red jagged lines seen below).
2. I run the point pairs through Matlab (actually Octave) to find the epipoles. Currently, I'm using Peter Kovesi's Peter's Functions for Computer Vision.
3. In Blender, I set up two cameras with the images overlaid. I orient the first camera based on the vanishing points. I also determine the focal lengths from the vanishing points. I orient the second camera relative to the first using the epipoles and one of the point pairs (below, the point at the top of the bandstand).
4. For each point pair, I project a ray from each camera through its sample point, and mark the closest covergence of the pair (in light yellow below). I realize that this leaves out information from the fundamental matrix - see below.

如您所见，这些点的融合程度不是很好.您从演奏台的位置向水平方向延伸的距离越远，左侧的效果越好.我猜想这表明了相机固有的差异.不幸的是，我找不到从点对应派生的 F 中找到内在函数的方法.

As you can see, the points don't converge very well. The ones from the left spread out the further you go horizontally from the bandstand point. I'm guessing that this shows differences in the camera intrinsics. Unfortunately, I can't find a way to find the intrinsics from an F derived from point correspondences.

最后，我认为我本身并不关心各个内在函数.我真正需要的是一种应用内部函数校正"图像的方法，以便将它们用作叠加层以手动优化模型.

In the end, I don't think I care about the individual intrinsics per se. What I really need is a way to apply the intrinsics to "correct" the images so that I can use them as overlays to manually refine the model.

这可能吗?我还需要其他信息吗?显然，我几乎没有希望找到有关相机内部特性的任何信息.尽管有一些明显的结构信息，例如哪些特征是正交的.我在某个地方看到了一个提示，即消失点可用于进一步优化或升级转换，但是我找不到任何具体的内容.

Is this possible? Do I need other information? Obviously, I have little hope of finding anything about the camera intrinsics. There is some obvious structural info though, such as which features are orthogonal. I saw a hint somewhere that the vanishing points can be used to further refine or upgrade the transformations, but I couldn't find anything specific.

更新1

Update 1

我可能已经找到了解决方案，但是在发布答案作为答案之前，我希望有人对此主题有所了解.事实证明，彼得的计算机视觉功能具有从采样点对单应性进行RANSAC估计的功能.使用m2 = H*m1，我应该能够在第二张图像上实际m2点的顶部绘制m1-> m2的映射.

I may have found a solution, but I'd like someone with some knowledge of the subject to weigh in before I post it as an answer. It turns out that Peter's Functions for Computer Vision has a function for doing a RANSAC estimate of the homography from the sample points. Using m2 = H*m1, I should be able to plot the mapping of m1 -> m2 over top of the actual m2 points on the second image.

唯一的问题是，我不确定我是否相信自己所看到的.即使在使用 F 的子极将图像对排列得很好的情况下，单应图的映射也看起来很糟糕.

The only problem is, I'm not sure I believe what I'm seeing. Even on an image pair that lines up pretty well using the epipoles from F, the mapping from the homography looks pretty bad.

我将尝试捕获可理解的图像，但是我的推理有什么问题吗?

I'll try to capture an understandable image, but is there anything wrong with my reasoning?

推荐答案

一些答案​​和建议(不分先后顺序):

A couple answers and suggestions (in no particular order):

1. 单应性仅在(a)摄像机进行纯旋转(不平移)或(b)相应的点都共面时正确地在点对应关系之间映射.
2. 基本矩阵仅涉及未校准的摄像机.从未知场景中恢复相机的校准参数(本征)的过程称为自动校准"，这是一个相当困难的问题.您需要这些参数(焦距，主要点)才能正确地重建场景.
3. 如果您拥有(许多)更多有关此场景的图像，则可以尝试使用Visual SFM之类的系统: http ://ccwu.me/vsfm/它将尝试自动解决结构自运动"问题，包括点匹配，自动校准和稀疏3D重建.

1. A homography will only correctly map between point correspondences when either (a) the camera undergoes a pure rotation (no translation) or (b) the corresponding points are all co-planar.
2. The fundamental matrix only relates uncalibrated cameras. The process of recovering a camera's calibration parameters (intrinsics) from unknown scenes, known as "auto-calibration" is a rather difficult problem. You'd need these parameters (focal length, principal point) to correctly reconstruct the scene.
3. If you have (many) more images of this scene, you could try using a system such as Visual SFM: http://ccwu.me/vsfm/ It will attempt to automatically solve the Structure From Motion problem, including point matching, auto-calibration and sparse 3D reconstruction.

这篇关于通过点对应检测/校正照片变形的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！