使用任意已知几何关系计算单应矩阵 [英] Calculating homography matrix using arbitrary known geometrical relations

问题描述

我正在将OpenCV用于光学测量系统。我需要在数码相机拍摄的两个图像之间进行透视转换。在摄像机的视场中，我放置了一组标记（位于同一平面上），它们用作两个图像中的对应点。使用标记的位置，我可以计算单应矩阵。问题在于，实际上要转换其图像的被测对象与标记之间的距离很小，并且与标记的平面平行。我可以测量这个距离。

我的问题是，在计算单应矩阵时如何考虑该距离，这对于执行透视变换是必不可少的。

在我的解决方案中，强烈要求不要使用测得的目标点来计算单应性（这就是为什么我在视场中需要其他标记）。

如果描述不准确，请告诉我。

在图中呈现的是示例性图像。

红色矩形是被测对象。它实际上被放置在圆形标记后面的一小段距离内。

我从不同相机的位置捕获物体的图像。在每次采集之间，被测对象可能会变形。我想使用圆形标记将对象的图像转换为相同的坐标。我可以测量对象和标记之间的距离，但是我不知道，如何修改单应性矩阵才能在被测对象（而不是标记）上工作。

解决方案

这个问题已经很老了，但是它很有趣并且对某些人可能有用。

首先，这是我对问题的理解：

您有两个图像I ₁ 和I ₂ 由同一台数码相机在两个不同的位置获取。这些图像都显示了一组标记，它们都位于同一平面p _m 中。还有一个被测物体，其可见表面位于平行于标记平面的p _o 平面中，但偏移很小。您计算了单应性H ^m ₁₂ ，将I ₁ 中的标记位置映射到I ₂ 中的相应标记位置然后您测量了平面p _o 和p _m 之间的偏移d _mo 。由此，您需要计算I ₁ 中被测对象的单应性H ^o ₁₂ 映射点到I < sub> 2 。

有关此问题的几点评论：

<首先请注意，单应性是图像点之间的关系，而标记平面与对象平面之间的距离是世界坐标中的距离。使用后者推断前者的某些情况需要对摄像机的姿势进行 metric 估算，即您需要确定 euclidian 和比例尺相对位置&相机针对两个图像的方向。 euclidian 要求意味着必须对数码相机进行校准，这对于光学测量系统来说应该不是问题。 按比例缩放要求意味着必须知道两个给定3D点之间的真实3D距离。例如，您需要知道两个任意标记之间的真实距离l ₀ 。

因为我们只需要 relative 1 使用与相机的坐标系居中并对齐的3D坐标系。因此，我们将通过P ₁ = K ₁ * [I | 0]。然后，我们通过P ₂ = K ₂ * [R ₂ | t ₂ ]。我们还将用D ₁ 和D ₂ 分别表示模拟I ₁ 和I ₂ 镜头畸变的系数

由于单个数码相机同时获取了I ₁ 和I ₂ ，因此您可以假定K ₁ = K ₂ = K且D ₁ = D ₂ =D。但是，如果I ₁ 和I ₂ 是在两次采集之间（或使用不同的缩放等）之间存在较长延迟的情况下获得的，因此考虑两个不同的相机矩阵和两组失真系数会更准确

以下是解决此类问题的方法：

估计P ₁ 和P ₂ 的步骤如下：

估计 ₁ ，k ₂ 和D ₁ ，D ₂



1. 使用D ₁ 和D ₂ 校正图像I ₁ 和I ₂ 处理镜头畸变，然后确定校正后图像中的标记位置




2. 计算基本矩阵F ₁₂ （将I ₁ 中的点映射到I ₂ 中的Epilines），并推断出基本矩阵E ₁₂ = K ₂ ^T * F ₁₂ * K ₁




3. 从E ₁₂ 推断出R ₂ 和t ₂ 以及一个点的对应关系（请参阅有关相关问题的答案）。在这一点上，您对摄像机的姿势有一个仿射估计，但由于t ₂ 具有单位范数，所以没有一个按比例放大的估计。




4. 使用两个任意标记之间的实测距离l ₀ 来推断t _{2 。}




5. 为获得最佳准确性，您可以使用捆绑包细化P ₁ 和P ₂ 调整，用K ₁ 和|| t ₂ ||固定，基于I ₁ 和I ₂ 中的相应标记位置。

在这一点上，您已经对摄像机姿态P ₁ = K ₁ *进行了精确的 metric 估计。 0]和P ₂ = K ₂ * [R ₂ | t ₂ ]。现在，估算H ^o ₁₂ 的步骤如下：

7. 使用D ₁ 和D ₂ 校正图像I ₁ 和I ₂ 镜头畸变，然后确定校正后图像中的标记位置（与上面的2.相同，无需重新进行此操作），并从这些对应位置估计H ^m ₁₂




8. 通过求解以下线性方程，计算描述标记平面p _m 的3x1向量v ₁₂ = K ₂ *（R ₂ -t ₂ * v ^T ）* K ₁ ^-1 （请参见HZ00第13章的结果13.5和公式13.2作为参考），其中Z是比例因子。推断到原点的距离d _m = || v ||和法线n = v / || v ||，它们以3D形式描述标记的平面p _m 。




9. 对象平面p _o 与p _m 平行，它们具有相同的法线n。因此，您可以从p _m

   到原点d _m 的距离推断出p _o 的原点d _o 的距离sub>和从测量的平面偏移d _mo ，如下所示：d _o = d _m ±d _mo （符号取决于平面的相对位置：如果p _m 对于I ₁ 比p _o 更靠近相机，则为正，为负




10. 从n和d _o 描述3D物体平面，推断出单应性H ^o ₁₂ = K ₂ *（R ₂ -t ₂ * n ^T / d _o ）* K ₁ ^-1 （请参见HZ00第13章的公式13.2）




11. 单应性H ^o ₁₂ 将I ₁ 中被测对象上的点映射到I _{1中的对应点sub> 2} ，其中I ₁ 和I ₂ 均已针对镜头畸变进行了校正。如果您需要将点映射到原始失真图像上，请不要忘记使用失真系数D ₁ 和D ₂ 来变换H ^o ₁₂ 。

我使用的参考：

[HZ00]计算机视觉的多视图几何，R.Hartley和A.Zisserman，2000年。

I am using OpenCV for an optical measurement system. I need to carry out a perspective transformation between two images, captured by a digital camera. In the field of view of the camera I placed a set of markers (which lie in a common plane), which I use as corresponding points in both images. Using the markers' positions I can calculate the homography matrix. The problem is, that the measured object, whose images I actually want to transform is positioned in a small distance from the markers and in parallel to the markers' plane. I can measure this distance.

My question is, how to take that distance into account when calculating the homography matrix, which is necessary to perform the perspective transformation.

In my solution it is a strong requirement not to use the measured object points for calculation of homography (and that is why I need other markers in the field of view).

Please let me know if the description is not precise.

Presented in the figure is the exemplary image.

The red rectangle is the measured object. It is physically placed in a small distance behind the circular markers.

I capture images of the object from different camera's positions. The measured object can deform between each acquisition. Using circular markers, I want to transform the object's image to the same coordinates. I can measure the distance between object and markers but I do not know, how should I modify the homography matrix in order to work on the measured object (instead of the markers).

解决方案

This question is quite old, but it is interesting and it might be useful to someone.

First, here is how I understood the problem presented in the question:

You have two images I₁ and I₂ acquired by the same digital camera at two different positions. These images both show a set of markers which all lie in a common plane p_m. There is also a measured object, whose visible surface lies in a plane p_o parallel to the marker's plane but with a small offset. You computed the homography H^m₁₂ mapping the markers positions in I₁ to the corresponding markers positions in I₂ and you measured the offset d_m-o between the planes p_o and p_m. From that, you would like to calculate the homography H^o₁₂ mapping points on the measured object in I₁ to the corresponding points in I₂.

A few remarks on this problem:

First, notice that an homography is a relation between image points, whereas the distance between the markers' plane and the object's plane is a distance in world coordinates. Using the latter to infer something about the former requires to have a metric estimation of the camera poses, i.e. you need to determine the euclidian and up-to-scale relative position & orientation of the camera for each of the two images. The euclidian requirement implies that the digital camera must be calibrated, which should not be a problem for an "optical measurement system". The up-to-scale requirement implies that the true 3D distance between two given 3D points must be known. For instance, you need to know the true distance l₀ between two arbitrary markers.

Since we only need the relative pose of the camera for each image, we may choose to use a 3D coordinate system centered and aligned with the coordinate system of the camera for I₁. Hence, we will denote the projection matrix for I₁ by P₁ = K₁ * [ I | 0 ]. Then, we denote the projection matrix for I₂ (in the same 3D coordinate system) by P₂ = K₂ * [ R₂ | t₂ ]. We will also denote by D₁ and D₂ the coefficients modeling lens distortion respectively for I₁ and I₂.

As a single digital camera acquired both I₁ and I₂, you may assume that K₁ = K₂ = K and D₁ = D₂ = D. However, if I₁ and I₂ were acquired with a long delay between the acquisitions (or with a different zoom, etc), it will be more accurate to consider that two different camera matrices and two sets of distortion coefficients are involved.

Here is how you could approach such a problem:

The steps in order to estimate P₁ and P₂ are as follows:

1. Estimate K₁, K₂ and D₁, D₂ via calibration of the digital camera

2. Use D₁ and D₂ to correct images I₁ and I₂ for lens distortion, then determine the marker positions in the corrected images

3. Compute the fundamental matrix F₁₂ (mapping points in I₁ to epilines in I₂) from the corresponding markers positions and infer the essential matrix E₁₂ = K₂^T * F₁₂ * K₁

4. Infer R₂ and t₂ from E₁₂ and one point correspondence (see this answer to a related question). At this point, you have an affine estimation of the camera poses, but not an up-to-scale one since t₂ has unit norm.

5. Use the measured distance l₀ between two arbitrary markers to infer the correct norm for t₂.

6. For the best accuracy, you may refine P₁ and P₂ using a bundle adjustment, with K₁ and ||t₂|| fixed, based on the corresponding marker positions in I₁ and I₂.

At this point, you have an accurate metric estimation of the camera poses P₁ = K₁ * [ I | 0 ] and P₂ = K₂ * [ R₂ | t₂ ]. Now, the steps to estimate H^o₁₂ are as follows:

7. Use D₁ and D₂ to correct images I₁ and I₂ for lens distortion, then determine the marker positions in the corrected images (same as 2. above, no need to re-do that) and estimate H^m₁₂ from these corresponding positions

8. Compute the 3x1 vector v describing the markers' plane p_m by solving this linear equation: Z * H^m₁₂ = K₂ * ( R₂ - t₂ * v^T ) * K₁^-1 (see HZ00 chapter 13, result 13.5 and equation 13.2 for a reference on that), where Z is a scaling factor. Infer the distance to origin d_m = ||v|| and the normal n = v / ||v||, which describe the markers' plane p_m in 3D.

9. Since the object plane p_o is parallel to p_m, they have the same normal n. Hence, you can infer the distance to origin d_o for p_o from the distance to origin d_m for p_m and from the measured plane offset d_m-o, as follows: d_o = d_m ± d_m-o (the sign depends of the relative position of the planes: positive if p_m is closer to the camera for I₁ than p_o, negative otherwise).

10. From n and d_o describing the object plane in 3D, infer the homography H^o₁₂ = K₂ * ( R₂ - t₂ * n^T / d_o ) * K₁^-1 (see HZ00 chapter 13, equation 13.2)

11. The homography H^o₁₂ maps points on the measured object in I₁ to the corresponding points in I₂, where both I₁ and I₂ are assumed to be corrected for lens distortion. If you need to map points from and to the original distorted image, don't forget to use the distortion coefficients D₁ and D₂ to transform the input and output points of H^o₁₂.

The reference I used:

[HZ00] "Multiple view geometry for computer vision", by R.Hartley and A.Zisserman, 2000.

这篇关于使用任意已知几何关系计算单应矩阵的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！