增强现实OpenGL + OpenCV [英] Augmented Reality OpenGL+OpenCV

问题描述

我是OpenCV的新手，在OpenGL方面的经验有限.我愿意将3D对象覆盖在棋盘的校准图像上.有任何提示或指导吗?

I am very new to OpenCV with a limited experience on OpenGL. I am willing to overlay a 3D object on a calibrated image of a checkerboard. Any tips or guidance?

推荐答案

基本概念是，您有2台摄像机:一台是物理摄像机(一台要使用opencv检索图像的摄像机)，一台是opengl摄像机.您必须对齐这两个矩阵.

The basic idea is that you have 2 cameras: one is the physical one (the one where you are retriving the images with opencv) and one is the opengl one. You have to align those two matrices.

为此，您需要校准物理摄像机.

To do that, you need to calibrate the physical camera.

首先.您需要一个失真参数(因为每个镜头或多或少都有一定的光学畸变)，并使用这些参数构建所谓的固有参数.为此，您可以在纸上打印一个棋盘，用它来获取一些图像并校准相机.它充满了关于Internet的不错的教程，从您的答案看来，您似乎已经拥有了它们.很好.

First. You need a distortion parameters (because every lens more or less has some optical distortion), and build with those parameters the so called intrinsic parameters. You do this with printing a chessboard in a paper, using it for get some images and calibrate the camera. It's full of nice tutorial about that on the internet, and from your answer it seems you have them. That's nice.

然后.您必须校准摄像机的位置.这是通过所谓的外部参数完成的.这些参数编码了这些相机的3D世界的位置和旋转.

Then. You have to calibrate the position of the camera. And this is done with the so called extrinsic parameters. Those parameters encoded the position and the rotation the the 3D world of those camera.

OpenCV方法cv::solvePnP和cv::Rodrigues需要内部参数，该方法使用rodrigues方法获取外部参数.该方法输入 2组对应点:一些3D知识点及其2D投影.这就是为什么所有增强现实应用程序都需要一些标记的原因:通常这些标记是方形的，因此在检测到它之后，您便知道点P1(0,0,0)P2(0,1,0)P3(1,1， 0)形成正方形的P4(1,0,0)，您可以找到位于它们上面的平面.

The intrinsic parameters are needed by the OpenCV methods cv::solvePnP and cv::Rodrigues and that uses the rodrigues method to get the extrinsic parameters. This method get in input 2 set of corresponding points: some 3D knowon points and their 2D projection. That's why all augmented reality applications need some markers: usually the markers are square, so after detecting it you know the 2D projection of the point P1(0,0,0) P2(0,1,0) P3(1,1,0) P4(1,0,0) that forms a square and you can find the plane lying on them.

一旦有了外部参数，便可以轻松解决所有游戏:您只需使用OpenGL的FoV和固有参数从相机的光圈角度在OpenGL中进行透视投影，然后将相机置于外部给定的位置参数.

Once you have the extrinsic parameters all the game is easily solved: you just have to make a perspective projection in OpenGL with the FoV and the aperture angle of the camera from intrinsic parameter and put the camera in the position given by the extrinsic parameters.

当然，如果您想要(并且应该)正确理解和处理此过程的每个步骤，那么..有很多数学-矩阵，角度，四元数，再次矩阵，以及..矩阵.您可以在著名的 多个R. Hartley和A.Zisserman的《计算机视觉中的几何学》 .

Of course, if you want (and you should) understand and handle each step of this process correctly.. there is a lot of math - matrices, angles, quaternion, matrices again, and.. matrices again. You can find a reference in the famous Multiple View Geometry in Computer Vision from R. Hartley and A. Zisserman.

此外，要正确处理opengl部分，您必须处理所谓的现代OpenGL"(记住，不推荐使用glLoadMatrix)和一些着色器，以加载相机位置的矩阵(对我而言，是个问题，因为我对此一无所知.

Moreover, to handle correctly the opengl part you have to deal with the so called "Modern OpenGL" (remember that glLoadMatrix is deprecated) and a little bit of shader for loading the matrices of the camera position (for me this was a problem because I didn't knew anything about it).

我之前已经处理过这个问题，并且我有一些代码，因此可以随时提出任何问题.这里有一些我感兴趣的链接:

I have dealt with this some times ago and I have some code so feel free to ask any kind of problems you have. Here some links I found interested:

1. http://ksimek.github.io/2012/08/14/分解/(很好的解释)
2. 摄像机在世界坐标中的位置，来自cv :: solvePnP (我问了一个问题)
3. http://www. morethantechnical.com/2010/11/10/20-lines-ar-in-opencv-wcode/(有关计算机视觉的神话般的博客)
4. http://spottrlabs.blogspot .it/2012/07/opencv-and-opengl-not-always-friends.html (妙招) http://strawlab.org/2011/11/05/augmented -reality-with-OpenGL/
5. http://www.songho.ca/opengl/gl_projectionmatrix.html (有关opengl相机设置基本知识的很好的解释)
6. 一些其他有用的东西: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction. html (文档，请始终查看文档！！！) 确定具有opencv的外部相机到具有世界空间的opengl对象 Rodrigues变成Eulerangles，反之亦然 Python Opencv SolvePnP产生错误的翻译向量 http://answers.opencv.org /question/23089/opencv-opengl-proper-camera-pose-using-solvepnp/

1. http://ksimek.github.io/2012/08/14/decompose/ (really good explanation)
2. Camera position in world coordinate from cv::solvePnP (a question I asked about that)
3. http://www.morethantechnical.com/2010/11/10/20-lines-ar-in-opencv-wcode/ (fabulous blog about computer vision)
4. http://spottrlabs.blogspot.it/2012/07/opencv-and-opengl-not-always-friends.html (nice tricks) http://strawlab.org/2011/11/05/augmented-reality-with-OpenGL/
5. http://www.songho.ca/opengl/gl_projectionmatrix.html (very good explanation on opengl camera settings basics)
6. some other random usefull stuffs: http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html (documentation, always look at the docs!!!) Determine extrinsic camera with opencv to opengl with world space object Rodrigues into Eulerangles and vice versa Python Opencv SolvePnP yields wrong translation vector http://answers.opencv.org/question/23089/opencv-opengl-proper-camera-pose-using-solvepnp/

请先阅读它们.像往常一样，一旦您理解了这个概念，这是一个简单的笑话，需要使您的大脑稍微撞到墙上.只是不要害怕所有这些数学运算:)

Please read them before anything else. As usual, once you got the concept it is an easy joke, need to crash your brain a little bit against the wall. Just don't be scared from all those math : )

这篇关于增强现实OpenGL + OpenCV的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！