增强现实OpenGL+OpenCV [英] Augmented Reality OpenGL+OpenCV

查看:37
本文介绍了增强现实OpenGL+OpenCV的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我对 OpenCV 非常陌生,对 OpenGL 的经验有限.我愿意在棋盘格的校准图像上覆盖 3D 对象.有什么提示或指导吗?

I am very new to OpenCV with a limited experience on OpenGL. I am willing to overlay a 3D object on a calibrated image of a checkerboard. Any tips or guidance?

推荐答案

基本的想法是你有 2 个摄像头:一个是物理的(你用 opencv 检索图像的那个),一个是 opengl 的.您必须对齐这两个矩阵.

The basic idea is that you have 2 cameras: one is the physical one (the one where you are retriving the images with opencv) and one is the opengl one. You have to align those two matrices.

为此,您需要校准物理相机.

To do that, you need to calibrate the physical camera.

首先.您需要一个畸变参数(因为每个镜头或多或少都有一些光学畸变),并使用这些参数构建所谓的内在参数.您可以通过在纸上打印棋盘来实现这一点,使用它来获取一些图像并校准相机.网上有很多关于这个的很好的教程,从你的回答看来你已经有了.挺好的.

First. You need a distortion parameters (because every lens more or less has some optical distortion), and build with those parameters the so called intrinsic parameters. You do this with printing a chessboard in a paper, using it for get some images and calibrate the camera. It's full of nice tutorial about that on the internet, and from your answer it seems you have them. That's nice.

那么.您必须校准相机的位置.这是通过所谓的外部参数完成的.这些参数编码了这些相机的 3D 世界的位置和旋转.

Then. You have to calibrate the position of the camera. And this is done with the so called extrinsic parameters. Those parameters encoded the position and the rotation the the 3D world of those camera.

OpenCV 方法cv::solvePnPcv::Rodrigues 需要内部参数,并且使用rodrigues 方法获取外部参数.该方法输入2组对应点:一些3D已知点及其2D投影.这就是为什么所有增强现实应用程序都需要一些标记的原因:通常标记是方形的,因此在检测到它之后,您就知道点 P1(0,0,0) P2(0,1,0) P3(1,1,0) P4(1,0,0) 形成一个正方形,你可以找到平面躺在上面.

The intrinsic parameters are needed by the OpenCV methods cv::solvePnP and cv::Rodrigues and that uses the rodrigues method to get the extrinsic parameters. This method get in input 2 set of corresponding points: some 3D knowon points and their 2D projection. That's why all augmented reality applications need some markers: usually the markers are square, so after detecting it you know the 2D projection of the point P1(0,0,0) P2(0,1,0) P3(1,1,0) P4(1,0,0) that forms a square and you can find the plane lying on them.

一旦你有了外部参数,所有的游戏就很容易解决了:你只需要在 OpenGL 中用 FoV 和来自内部参数的相机的孔径角进行透视投影,然后将相机放在外部参数给定的位置参数.

Once you have the extrinsic parameters all the game is easily solved: you just have to make a perspective projection in OpenGL with the FoV and the aperture angle of the camera from intrinsic parameter and put the camera in the position given by the extrinsic parameters.

当然,如果你想(并且你应该)正确理解和处理这个过程的每一步......有很多数学 - 矩阵,角度,四元数,矩阵,和..矩阵.您可以在著名的 Multiple View 中找到参考计算机视觉中的几何来自 R. Hartley 和 A. Zisserman.

Of course, if you want (and you should) understand and handle each step of this process correctly.. there is a lot of math - matrices, angles, quaternion, matrices again, and.. matrices again. You can find a reference in the famous Multiple View Geometry in Computer Vision from R. Hartley and A. Zisserman.

此外,要正确处理 opengl 部分,您必须处理所谓的现代 OpenGL"(请记住,glLoadMatrix 已弃用)和一些用于加载相机矩阵的着色器位置(对我来说这是一个问题,因为我对此一无所知).

Moreover, to handle correctly the opengl part you have to deal with the so called "Modern OpenGL" (remember that glLoadMatrix is deprecated) and a little bit of shader for loading the matrices of the camera position (for me this was a problem because I didn't knew anything about it).

我以前处理过这个问题,我有一些代码,所以请随时提出您遇到的任何问题.这里有一些我感兴趣的链接:

I have dealt with this some times ago and I have some code so feel free to ask any kind of problems you have. Here some links I found interested:

  1. http://ksimek.github.io/2012/08/14/decompose/(很好的解释)
  2. 来自 cv::solvePnP 的世界坐标中的相机位置(我问过的一个问题)
  3. http://www.morethantechnical.com/2010/11/10/20-lines-ar-in-opencv-wcode/(关于计算机视觉的精彩博客)
  4. http://spottrlabs.blogspot.it/2012/07/opencv-and-opengl-not-always-friends.html(好技巧)http://strawlab.org/2011/11/05/augmented-现实与 OpenGL/
  5. http://www.songho.ca/opengl/gl_projectionmatrix.html (关于opengl相机设置基础的很好的解释)
  6. 其他一些随机有用的东西:http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html(文档,总是看文档!!!)用opencv确定外在相机到世界空间的opengl对象罗德里格斯到欧拉角,反之亦然Python Opencv SolvePnP 产生错误的翻译向量http://answers.opencv.org/问题/23089/opencv-opengl-proper-camera-pose-using-solvepnp/
  1. http://ksimek.github.io/2012/08/14/decompose/ (really good explanation)
  2. Camera position in world coordinate from cv::solvePnP (a question I asked about that)
  3. http://www.morethantechnical.com/2010/11/10/20-lines-ar-in-opencv-wcode/ (fabulous blog about computer vision)
  4. http://spottrlabs.blogspot.it/2012/07/opencv-and-opengl-not-always-friends.html (nice tricks) http://strawlab.org/2011/11/05/augmented-reality-with-OpenGL/
  5. http://www.songho.ca/opengl/gl_projectionmatrix.html (very good explanation on opengl camera settings basics)
  6. some other random usefull stuffs: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html (documentation, always look at the docs!!!) Determine extrinsic camera with opencv to opengl with world space object Rodrigues into Eulerangles and vice versa Python Opencv SolvePnP yields wrong translation vector http://answers.opencv.org/question/23089/opencv-opengl-proper-camera-pose-using-solvepnp/

请先阅读它们.像往常一样,一旦你有了这个概念,这就是一个简单的笑话,需要让你的大脑稍微撞到墙上.只是不要害怕所有这些数学:)

Please read them before anything else. As usual, once you got the concept it is an easy joke, need to crash your brain a little bit against the wall. Just don't be scared from all those math : )

这篇关于增强现实OpenGL+OpenCV的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆