世界中给定 Z 的 2D 到 3D 投影 [英] 2D to 3D Projection with given Z in world

问题描述

如果之前有人问过，我很抱歉，但我找不到我问题的正确答案.

I'm sorry if it has been ask before but I couldn't find the proper answer to my question.

为了更好地理解，让我简要解释一下我的问题的背景

For a better understanding, let me briefly explain the context of my problem

背景

我有两个图像(A 和 B)，上面有非平面物体.我希望能够从 A 中获取像素 pA 的坐标并将其投影到 B 中.由于我的场景不是平面的，我不能使用单应性.我想要做的是首先将我的像素 pA 投影到 3D 世界中，然后将结果投影到图像 B 中以获得 pB.pA (2D) -> pWorld (3D) -> pB (2D).幸运的是，我知道 pworld 的坐标 z.我的问题涉及第一步 pA (2D) -> pWorld (3D).

I have two images (A and B) with non planar object on it. I would like to be able to take the coordinate of a pixel pA from A and project it into B. Since my scene is not planar, I can't use homography. What I want to do is first project my pixel pA into the 3D world and then project the result into the image B to get pB. pA (2D) -> pWorld (3D) -> pB (2D). Fortunately, I know the coordinate z of pworld. My question concerns the first step pA (2D) -> pWorld (3D).

问题

如何将我的二维点 pA (u,v) 投影到世界中 (pWorld=(X,Y,Z))，给出 Z?我还有我的相机的外在矩阵 Rt (3x4) 和内在矩阵 K (3x3).

How to project my 2D point pA (u,v) into the world (pWorld=(X,Y,Z)), Z being given? I also have the extrinsic matrix Rt (3x4) and the intrinsic matrix K (3x3) of my camera.

我尝试了什么

我知道:s*(u v 1)' = K * Rt * (X Y Z)' [1]

s 是比例.但是我想有相反的过程，z报错.类似的东西:

s is the scale. But I would like to have the opposite process, Z being given. Something like:

(X Y) = SOMETHING * (u v)

我可以重写 [1] 以获得s*(u v 1/s 1/s)' = G * (XY Z 1)'

I can rewrite [1] to get s*(u v 1/s 1/s)' = G * (X Y Z 1)'

G = (l1 l2 l3 l4)(l 表示线)

with G = (l1 l2 l3 l4) (l means line)

l1 = (K*Rt) 的第一行

l1 = first line of (K*Rt)

l2 = (K*Rt) 的第二行

l2 = second line of (K*Rt)

l3 = 0 0 1/Z 0

l3 = 0 0 1/Z 0

l4 = 0 0 0 1

l4 = 0 0 0 1

G 是可逆的，然后我可以有(XY Z 1)' = inv(G) * (us vs 1 1)'

G is invertible and I can then have (X Y Z 1)' = inv(G) * (us vs 1 1)'

但我不能使用它，因为我不知道比例.我想我对这个规模的事情有点困惑.我知道通常我们会通过规范化来摆脱它，但在这里，我不能.

But I can't use that since I don't know the scale. I think I'm a bit confused concerning this scale thing. I know usually we normalized to get rid of it but here, I can't.

也许这不是继续的好方法.如果有人能以好的方式向我解释，我会很高兴听到这个消息.

Maybe that's not the good way to proceed. If someone can explain me the good way, I would be really glad to hear about it.

提前致谢.

推荐答案

我找到了一个解决方案，但它太丑了.

I found a solution but it is damn ugly.

让我们考虑 3x4 矩阵 M:

Let's consider the 3x4 matrix M:

M = K*Rt = (mij) 1<i<3, 1<j<4

为了简单起见，我们还考虑系数 A 和 B:

For simplification, let's also consider the coefficients A and B:

A = (m12-m32*u)/(m22-m32v)
B = (m31*u-m11)/(m31*v-m21)

符号解释完毕，让我们继续介绍系统.正如我所说，系统是:

The notation explained, let's move on to the system. As I said, the system is:

s*(u v 1)' = M*(X Y Z 1)'

我们有3个方程和3个未知数:s、X和Y.我们可以注意到:

We have 3 equations and 3 unknowns : s, X and Y. We can notice that:

s = m31*X + m32*Y + m33*Z + m34

注意，如果你想投影到相机坐标系而不是世界坐标系(类似于没有旋转和平移的情况)，你有 s = Z 这是一个更简单的系统解决(这里的例子 计算世界坐标从使用 OpenCV 的屏幕坐标)

考虑到这一点，我们可以将原始系统简化为具有 2 个未知数(X 和 是):

With this in mind, we can reduce the original system into a system of 2 equations with 2 unknowns (X and Y):

然后，经过一番计算，我们终于得到:

Then, after some calculations, we finally get:

X = [Z*((m23-M33*v)*A-m13+m33*u) + (m24-m34*v)*A-m14+m34*u ] / [A*(m31*v-m21)-m31*u+m11]

Y = [Z*((m13-m33*u)-B*(m23-m33*v)) + m14-m34*u-B*(m24-m34*v)] / [B*(m22-m32*v)-m12+m32*u]

它是 X 和 Y 在 u、v 和 Z 的函数中的表达式.我用我的项目测试了它，它工作正常.

It is the expression of X and Y in function of u, v and Z. I tested that with my project and it was working.

不知道是否有更简洁的方法来计算(使用 Matrix 和其他东西)，但我现在只能想到这些.

Don't know if there is a cleaner way to compute that (with Matrix and stuff), but that's all I could come up with for now.

这篇关于世界中给定 Z 的 2D 到 3D 投影的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！