计算单个平面上多个标记的单应性 [英] Calculating homography of multiple markers on a single plane

问题描述

我有一个算法使用OpenCV的 findHomography 计算单个图像中多个标记的单应性。这工作正常，但由于图像的标记/分辨率的大小，计算的旋转通常是非常不准确的（主要围绕标记的x和z轴，假设y向上）。我想找到一种方法来改善这一点。

我有一个假设，我知道所有的标记都在同一个平面上（他们在一个表）应该有助于改善单应性 - 我只需要解决标记围绕单个轴的旋转 - 但我不知道足够把这个练习。任何人都可以给我任何指导这个吗？

这是一个例子，我想处理的情况（标记上的图像在现实中是不同的）。

>

解决方案

我喜欢通过使用跨场景的多个标记图像来获得更高精度的估计单应性。

这种方法可以使用非线性优化实现。

假设，模型，数据

让我们以3个打印的标记图像为例

现在让我们假设对于3个打印的标记图像中的每一个，您都找到了 N

点对应关系表示对于打印的标记图像，您可以将 N 图片像素位置与 N 现实世界位置相对于已知的真实世界打印的标记坐标系（例如，坐标（0,0）），以厘米（或一些其他单位） ）

问题是，打印的标记坐标系仅在每个打印标记图像。我们需要的是将这些坐标系统结合在一起，并且找到一种在单个全局坐标系统中表达印刷标记坐标系位置的方式。

每个标记的单应性之间的关系

我们要计算使用来自不同打印标记图像的所有对应数据的单应性。因此，重要的是要了解它们如何与每个相关。具体来说，我们想为所有标记图像找到一个共同的坐标系，以便制定一个优化问题。

现在，根据点对应关系，计算3个单应性：

• H1 图像＃1到图像像素坐标


• H2 将标记图像＃2的打印标记坐标系 / li>
  
• H3 将标记图像＃3的打印标记坐标系位置映射到图像像素坐标

让我们很快假设没有测量和计算错误（只是假设）。然后，单应性只在旋转和翻译。

然后，以下等式成立：

，其旋转和平移定义如下：

/ h2>

这意味着H1等于H2直到旋转和翻译和翻译

这给我们提供了一种将标记图像＃2的位置转换为标记图像＃1的坐标系统的方法（对于标记图像＃3的位置类似）：

现在，在现实世界的场景中，只有大约。但是它们给出了估计任何一对打印的标记图像之间的旋转和平移的动机：

href =https://en.wikipedia.org/wiki/Atan2 =nofollow> atan2 ，我们可以找到六个辅助变量

这是我们需要准备的起始值和一个模型来制定非线性优化问题。

非线性优化问题

最后的目标是找到单应矩阵<8>的系数 H ，其使用所有点对应所有已打印的标记图片：

上述模型中，我们可以定义一个非线性方程组。将有 8 + 6 未知数：

• 8个单应系数


• 6个辅助变量，其描述如何将标记图像＃2和＃3的对应关系映射到标记图像＃1坐标系。

使用3个标记图像的 N 点对应，我们得到 N * 3 * 2 = N * 6 方程。这意味着，为了具有比未知数更多的方程，我们需要 N> = 3 点对应。有趣的是，在多标记图像情形下，每个标记图像的最小 N = 3 对应小于最小 N = 4 在单个标记图像场景的情况下。

我不会在这一点更详细地添加非线性优化问题的公式作为一般想法

为了解决非线性优化问题， Newton's method

备注

上述等式中的所有矩阵都需要按如下方式进行规范化：

I've got an algorithm that calculates the homography of a number of markers in a single image, using OpenCV's findHomography. This works fine, but due to the size of the markers/resolution of the images, the rotation calculated is often quite inaccurate (mainly around the x and z axes of the marker, assuming y is up). I'd love to find a way to improve this.

I have a hypothesis that the fact that I know all the markers are on the same plane (they're on a table) should help improve the homography - I then just need to work out the rotation of the marker around a single axis - but I don't know enough to put this in to practice. Can anyone give me any guidance on this?

This is an example of the situation I'm trying to deal with (the images on the markers are different in reality).

解决方案

I like the idea of getting higher precision for the estimated homography by using multiple marker images spread across the scene.

This approach can be realized using Nonlinear Optimization. But it’s a little tricky.

Assumptions, Model, Data

Let’s take your example of 3 printed marker images placed across a table and assume all your assumptions hold.

And now let’s assume that for each of the 3 printed marker images you’ve found N point correspondences.

Point correspondences here means that for a printed marker image you can associate N image pixel positions with corresponding N real world positions (x,y) in centimeters (or some other unit) with respect to a known real world "printed marker coordinate system" (for example, with coordinate (0,0) located at the center of a marker image).

The problem is that the "printed marker coordinate system" is only known locally for each printed marker image. What we need is to bring those coordinate systems together and find a way to express "printed marker coordinate system" positions in a single, global coordinate system. So we need to figure out a relation between them.

Relation between each marker’s homographies

We want to compute a homography which uses all the correspondence data from different printed marker images. Therefore, it is important to understand how they relate to each. Specifically, we want to find a common coordinate system for all marker images in order to formulate an optimization problem.

Now, based on the point correspondences, and using conventional methods, compute 3 homographies:

• H1 maps "printed marker coordinate system" positions of marker image #1 to image pixel coordinates
• H2 maps "printed marker coordinate system" positions of marker image #2 to image pixel coordinates
• H3 maps "printed marker coordinate system" positions of marker image #3 to image pixel coordinates

Let’s assume shortly that no measurement and computation errors are present (just hypothetically). Then, the homographies differ only in a rotation and translation of the respective "printed marker coordinate systems".

Then, the following equations hold:

with the rotation and translation defined as follows:

Putting everything together

That means that H1 equals H2 "up to" a rotation and translation and H1 equals H3 "up to" a rotation and translation .

This gives us a way for transforming positions of marker image #2 into the coordinate system of marker image #1 (case for positions of marker image #3 can be handled analogous):

Now, in a real world scenario, these equations will hold only approximately. But nevertheless they give a motivation for estimating the rotation and translation between any pair of printed marker images:

Now, using atan2, we can find estimates of six auxiliary variables which describe how the printed marker images #2 and #3 are located relative to #1.

That’s all we need for preparation to have start values and a model to formulate a nonlinear optimization problem.

Nonlinear optimization problem

The final goal is to find 8 coefficients of a homography matrix H which uses all point correspondences from all printed marker images at the same time:

Using the above model, we can define a system of nonlinear equations. It will have 8 + 6 unknowns:

• 8 homography coefficients
• 6 auxiliary variables which describe how the correspondences from marker image #2 and #3 will be mapped to the marker image #1 coordinate system

Using the N point correspondences of 3 marker images, we get N*3*2 = N*6 equations. That means, we need N>=3 point correspondences in order to have more equations than unknowns. Interestingly, the minimum N=3 correspondences per marker image in case of the multiple marker image scenario is less than the minimum N=4 in the case of a single marker image scenario.

I’ll not add the formulation of the nonlinear optimization problem in more detail at this point as the general idea of how to approach the problem was already explained.

For solving the nonlinear optimization problem, Newton’s method can be used, for example.

Remark

All matrices in the above equations need to be normalized like follows:

这篇关于计算单个平面上多个标记的单应性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！