带有OpenCV的增强现实SDK [英] Augmented Reality SDK with OpenCV
问题描述
我正在OpenCV上开发增强现实SDK.我在查找有关该主题的教程时遇到了一些问题，需要遵循的步骤，可能的算法，实时性能的快速高效编码等.
I am developing an Augmented Reality SDK on OpenCV. I had some problems to find tutorials on the topic, which steps to follow, possible algorithms, fast and efficient coding for real-time performance etc.
到目前为止，我已经收集了以下信息和有用的链接.
So far I have gathered the next information and useful links.
下载最新发行版.
您可以在此处中找到安装指南(平台:linux ，mac，windows，java，android，iOS).
You can find installation guides here (platforms: linux, mac, windows, java, android, iOS).
在线文档.
对于初学者此处是OpenCV中的一个简单的增强现实代码.这是一个好的开始.
For begginers here is a simple augmented reality code in OpenCV. It is a good start.
对于正在寻找设计良好的最新SDK的任何人，我发现了一些考虑到OpenCV功能的基于标记跟踪的增强现实应该具有的常规步骤.
For anyone searching for a well designed state-of-the-art SDK I found some general steps that every augmented-reality based on marker tracking should have, considering OpenCV functions.
-
主程序:创建所有类，初始化，从视频中捕获帧.
AR_Engine类:控制增强现实应用程序的各个部分.应该有2个主要状态:
AR_Engine class: Controls the parts of an augmented reality application. There should be 2 main states:
- 检测:尝试检测场景中的标记
- 跟踪:一旦检测到，将使用较低的计算技术来跟踪即将到来的帧中的标记.
- detection: tries to detect the marker in the scene
- tracking: once it is detected, uses lower computational techniques for traking the marker in upcoming frames.
还应该有一些算法可以在每帧中找到相机的位置和方向.这是通过检测场景中检测到的标记与我们离线处理的标记的2D图像之间的单应性变换来实现的.有关此方法的说明，请此处(第18页).姿势估计的主要步骤是:
Also there should be some algorithms for finding the position and orientation of the camera in every frame. This is achieve by detecting the homography transformation between the marker detected in the scene, and a 2D image of the marker we have processed offline. The explanation of this method here (page 18). The main steps for Pose Estimations are:
-
加载相机固有参数.以前通过校准离线提取.
Load camera Intrinsic Parameters. Previously extracted offline through calibration.
加载图案(标记)以进行跟踪:这是我们要跟踪的平面标记的图像.有必要提取特征并生成描述符(关键点)这种模式，以便稍后我们可以与场景中的特征进行比较.此任务的算法:
Load the pattern (marker) to track: It is an image of the planar marker we are going to track. It is necessary to extract features and generate descriptors (keypoints) for this pattern so later we can compare with features from the scene. Algorithms for this task:
- SIFT
- FAST
- SURF
对于每次更新帧，运行检测算法以从场景中提取特征并生成描述符.同样，我们有几个选择.
For every frame update, run a detection algorithm for extracting features from the scene and generate descriptors. Again we have several options.
- SIFT
- FAST
- SURF
- FREAK: A new method (2012) supossed to be the fastest.
- ORB
找到图案和场景描述符之间的匹配项.
Find matches between pattern and the scene descriptors.
从那些匹配项中找到 Homography 矩阵.可以先使用RANSAC来找到匹配项中的孤立点.
Find Homography matrix from those matches. RANSAC can be used before to find inliers/outliers in the set of matches.
从单应图中提取相机姿势.
- Homography的姿势上的示例代码.
- Pose的Homography 上的示例代码.
- Sample code on Pose from Homography.
- Sample code on Homography from Pose.
完整示例:
- aruco
- 掌握OpenCV示例
