电影坐标到世界坐标 [英] Film coordinate to world coordinate

问题描述

我正在使用 OpenCV3.1 和 OpenGL 从特征匹配构建 3D 点云.

我已经实现了 1) 相机校准(因此我有相机的内在矩阵)2)特征提取(因此我在像素坐标中有 2D 点).我浏览了几个网站，但通常都建议将 3D 对象点转换为像素点的流程，但我正在做完全反向投影.这是

每个相机都有分辨率 xs,ys 和由两个角度定义的视野 FOV FOVx,FOVy，因此请查看您的相机数据表或测量它.从该距离和垂直距离 (z)，您可以将任何像素位置 (x,y) 转换为相对于相机的 3D 坐标 (x',y',z').所以首先将像素位置转换为角度:

ax = (x - (xs/2)) * FOVx/xsay = (y - (ys/2)) * FOVy/ys

然后计算 3D 中的笛卡尔位置:

x' = distance * tan(ax)y' = 距离 * tan(ay)z' = 距离

这很好，但在普通图像上我们不知道距离.幸运的是，在这样的设置中，如果我们转动我们的物体，那么如果与相机的垂直平面相交，任何凸边都会在侧面形成最大的 ax 角度.因此，检查几帧，如果检测到最大 ax，您可以假设它的边缘(或凸块)位于 distance 处.

如果您还知道平台的旋转角度 ang(相对于您的相机)，那么您可以使用

因此检查所有帧上的一条水平线并找到最大的ax.为了提高准确性，您可以通过转动平台直到准确地"找到峰值来执行闭环调节循环.分别对所有水平线执行此操作.

顺便说一句.如果您在几帧内没有检测到 ax 变化，这意味着具有相同半径的圆形......因此您可以将每个这样的帧处理为 ax 最大值.

像馅饼一样简单，生成 3D 点云.您可以按平台角度排序以简化向网格的转换...该角度也可以用作纹理坐标...

但是不要忘记，你会丢失一些隐藏在剪影中的凹面细节！！！

如果这种方法还不够，您可以使用相同的设置进行立体 3D 重建.因为每次旋转都表现为新的(已知)相机位置.

I am working on building 3D point cloud from features matching using OpenCV3.1 and OpenGL.

I have implemented 1) Camera Calibration (Hence I am having Intrinsic Matrix of the camera) 2) Feature extraction( Hence I have 2D points in Pixel Coordinates). I was going through few websites but generally all have suggested the flow for converting 3D object points to pixel points but I am doing completely backword projection. Here is the ppt that explains it well.

I have implemented film coordinates(u,v) from pixel coordinates(x,y)(With the help of intrisic matrix). Can anyone shed the light on how I can render "Z" of camera coordinate(X,Y,Z) from the film coordinate(x,y).

Please guide me on how I can utilize functions for the desired goal in OpenCV like solvePnP, recoverPose, findFundamentalMat, findEssentialMat.

解决方案

With single camera and rotating object on fixed rotation platform I would implement something like this:

Each camera has resolution xs,ys and field of view FOV defined by two angles FOVx,FOVy so either check your camera data sheet or measure it. From that and perpendicular distance (z) you can convert any pixel position (x,y) to 3D coordinate relative to camera (x',y',z'). So first convert pixel position to angles:

ax = (x - (xs/2)) * FOVx / xs 
ay = (y - (ys/2)) * FOVy / ys 

and then compute cartesian position in 3D:

x' = distance * tan(ax)
y' = distance * tan(ay)
z' = distance

That is nice but on common image we do not know the distance. Luckily on such setup if we turn our object than any convex edge will make an maximum ax angle on the sides if crossing the perpendicular plane to camera. So check few frames and if maximal ax detected you can assume its an edge (or convex bump) of object positioned at distance.

If you also know the rotation angle ang of your platform (relative to your camera) Then you can compute the un-rotated position by using rotation formula around y axis (Ay matrix in the link) and known platform center position relative to camera (just subbstraction befor the un-rotation)... As I mention all this is just simple geometry.

In an nutshell:

1. obtain calibration data

   FOVx,FOVy,xs,ys,distance. Some camera datasheets have only FOVx but if the pixels are rectangular you can compute the FOVy from resolution as

   FOVx/FOVy = xs/ys
   

   Beware with Multi resolution camera modes the FOV can be different for each resolution !!!

2. extract the silhouette of your object in the video for each frame

   you can subbstract the background image to ease up the detection

3. obtain platform angle for each frame

   so either use IRC data or place known markers on the rotation disc and detect/interpolate...

4. detect ax maximum

   just inspect the x coordinate of the silhouette (for each y line of image separately) and if peak detected add its 3D position to your model. Let assume rotating rectangular box. Some of its frames could look like this:

   So inspect one horizontal line on all frames and found the maximal ax. To improve accuracy you can do a close loop regulation loop by turning the platform until peak is found "exactly". Do this for all horizontal lines separately.

   btw. if you detect no ax change over few frames that means circular shape with the same radius ... so you can handle each of such frame as ax maximum.

Easy as pie resulting in 3D point cloud. Which you can sort by platform angle to ease up conversion to mesh ... That angle can be also used as texture coordinate ...

But do not forget that you will lose some concave details that are hidden in the silhouette !!!

If this approach is not enough you can use this same setup for stereoscopic 3D reconstruction. Because each rotation behaves as new (known) camera position.

这篇关于电影坐标到世界坐标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！