电影坐标到世界坐标 [英] Film coordinate to world coordinate

问题描述

我正在通过使用OpenCV3.1和OpenGL进行功能匹配来构建3D点云.

I am working on building 3D point cloud from features matching using OpenCV3.1 and OpenGL.

我已经实现了1)相机校准(因此，我具有相机的本征矩阵)2)特征提取(因此，我在像素坐标中具有2D点). 我浏览的网站很少，但总体上都提出了将3D对象点转换为像素点的流程，但是我正在做完全背单词投影.这是 ppt 对其进行了很好的解释.

I have implemented 1) Camera Calibration (Hence I am having Intrinsic Matrix of the camera) 2) Feature extraction( Hence I have 2D points in Pixel Coordinates). I was going through few websites but generally all have suggested the flow for converting 3D object points to pixel points but I am doing completely backword projection. Here is the ppt that explains it well.

我已经从像素坐标(x，y)实现了影片坐标(u，v)(借助内在矩阵).谁能阐明我如何从胶片坐标(x，y)渲染摄像机坐标(X，Y，Z)的"Z".

I have implemented film coordinates(u,v) from pixel coordinates(x,y)(With the help of intrisic matrix). Can anyone shed the light on how I can render "Z" of camera coordinate(X,Y,Z) from the film coordinate(x,y).

请指导我如何在OpenCV中实现期望目标的功能，例如SolvePnP，recoverPose，findFundamentalMat，findEssentialMat.

Please guide me on how I can utilize functions for the desired goal in OpenCV like solvePnP, recoverPose, findFundamentalMat, findEssentialMat.

推荐答案

在固定旋转平台上使用单个摄像机和旋转对象，我将实现以下内容:

With single camera and rotating object on fixed rotation platform I would implement something like this:

每个摄像机的分辨率为xs,ys，视场 FOV 由两个角度 FOVx，FOVy 定义，因此请检查或测量您的摄像机数据表.从该距离和垂直距离(z)，您可以将任何像素位置(x,y)转换为相对于相机(x'，y'，z')的3D坐标.因此，首先将像素位置转换为角度:

Each camera has resolution xs,ys and field of view FOV defined by two angles FOVx,FOVy so either check your camera data sheet or measure it. From that and perpendicular distance (z) you can convert any pixel position (x,y) to 3D coordinate relative to camera (x',y',z'). So first convert pixel position to angles:

ax = (x - (xs/2)) * FOVx / xs 
ay = (y - (ys/2)) * FOVy / ys 

然后在3D中计算笛卡尔位置:

and then compute cartesian position in 3D:

x' = distance * tan(ax)
y' = distance * tan(ay)
z' = distance

很好，但是根据普通图像，我们不知道distance.幸运的是，在这种设置下，如果我们将对象旋转，则如果与摄影机垂直的平面相交，则任何凸边都将在侧面形成最大ax角度.因此，检查几帧，如果检测到最大ax，则可以假定其位于distance处的对象的边缘(或凸块).

That is nice but on common image we do not know the distance. Luckily on such setup if we turn our object than any convex edge will make an maximum ax angle on the sides if crossing the perpendicular plane to camera. So check few frames and if maximal ax detected you can assume its an edge (or convex bump) of object positioned at distance.

如果您还知道平台(相对于相机)的旋转角度ang，则可以使用

If you also know the rotation angle ang of your platform (relative to your camera) Then you can compute the un-rotated position by using rotation formula around y axis (Ay matrix in the link) and known platform center position relative to camera (just subbstraction befor the un-rotation)... As I mention all this is just simple geometry.

简而言之:

1. 获取校准数据

FOVx，FOVy，xs，ys，距离.某些相机数据表仅包含FOVx，但是如果像素为矩形，则可以根据以下分辨率计算FOVy

FOVx,FOVy,xs,ys,distance. Some camera datasheets have only FOVx but if the pixels are rectangular you can compute the FOVy from resolution as

FOVx/FOVy = xs/ys

当心使用多分辨率相机模式时，每种分辨率的FOV可能不同！！

Beware with Multi resolution camera modes the FOV can be different for each resolution !!!

针对每一帧提取视频中对象的轮廓

您可以减去背景图像以简化检测

you can subbstract the background image to ease up the detection

获取每帧的平台角度

因此可以使用 IRC 数据或将已知标记放置在转盘上并进行检测/内插...

so either use IRC data or place known markers on the rotation disc and detect/interpolate...

检测到最大ax

detect ax maximum

只需检查轮廓的x坐标(分别针对图像的每个y线)，如果检测到峰，则将其 3D 位置添加到模型中.假设旋转矩形框.它的某些框架可能看起来像这样:

just inspect the x coordinate of the silhouette (for each y line of image separately) and if peak detected add its 3D position to your model. Let assume rotating rectangular box. Some of its frames could look like this:

因此，检查所有帧上的一条水平线，找到最大的ax.为了提高精度，您可以通过旋转平台直至精确"找到峰值来进行闭环调节.分别对所有水平线执行此操作.

So inspect one horizontal line on all frames and found the maximal ax. To improve accuracy you can do a close loop regulation loop by turning the platform until peak is found "exactly". Do this for all horizontal lines separately.

顺便说一句..如果您未检测到ax在几帧中都没有变化，这意味着半径相同的圆形...因此，您可以将每个这样的帧处理为ax最大值.

btw. if you detect no ax change over few frames that means circular shape with the same radius ... so you can handle each of such frame as ax maximum.

易于制作饼图，从而生成3D点云.您可以按平台角度对其进行排序，以简化向网格的转换...该角度也可以用作纹理坐标...

Easy as pie resulting in 3D point cloud. Which you can sort by platform angle to ease up conversion to mesh ... That angle can be also used as texture coordinate ...

但是不要忘记，您会丢失一些隐藏在轮廓中的凹形细节！！！

But do not forget that you will lose some concave details that are hidden in the silhouette !!!

如果这种方法还不够，您可以使用相同的设置进行立体3D重建.因为每次旋转都表现为新的(已知)相机位置.

If this approach is not enough you can use this same setup for stereoscopic 3D reconstruction. Because each rotation behaves as new (known) camera position.

这篇关于电影坐标到世界坐标的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！