像zookazam一样增强现实 [英] Augment reality like zookazam

问题描述

1. 增强现实使用了哪些算法，例如

   我认为它可以通过对比分析图像并找到平面，但我不知道如何.

2. 在开始使用此类应用之前，我应该阅读哪些主题?

解决方案

[Prologue]

这是一个非常广泛的话题，并且在当前状态下大部分是离题.我重新编辑了您的问题，但为了使您的问题在本网站的规则/可能性范围内可以回答

您应该更详细地说明您的增强现实:

1. 应该做的

   • 添加具有已知网格的 2D/3D 对象...
   • 不断变化的光照条件
   • 添加/移除身体部位/衣服/头发......

   一个好主意是提供一些您想要实现的输入/输出的示例图像(草图).

2. 它有什么输入

   • 视频、静态图像、2D、立体、3D.对于纯 2D 输入，请指定您必须帮助重建的条件/标记/照明/激光 模式.
   • 输入图像中会有什么?空房间、人、特定物体等.

3. 指定目标平台

   许多算法受限于内存大小/带宽、CPU 能力、特殊的硬件 功能等，因此最好为您的平台添加标签.操作系统和语言也是一个好主意.

[增强现实的工作原理]

1. 获取输入图像

   如果您要连接到诸如相机之类的设备，则需要使用其驱动程序/框架或其他东西来获取图像或使用它支持的一些常见API.此任务依赖于操作系统.我在 Windows 上最喜欢的方法是使用 VFW(Windows 视频)API.

   我会从一开始就从一些静态文件开始，以简化调试和增量构建过程.(您不需要等待相机和每次构建时发生的事情).当您的应用准备好进行实时视频播放时，请切换回相机...

2. 将场景重建为 3D 网格

   如果您使用 3D 相机(如 Kinect)，则无需执行此步骤.否则，您需要通过一些分割过程来区分对象，通常基于边缘检测或颜色均匀性.

   3D 网格的质量取决于您想要实现的目标和您的输入.例如，如果您想要逼真的阴影和照明，那么您需要非常好的网格.如果相机固定在某个房间，您可以手动预定义网格(硬编码)并仅计算视图中的对象.此外，通过从当前视图图像中减去空房间图像，可以非常简单地完成对象检测/分割，因此差异较大的像素就是对象.

   您也可以按照 OP 中的建议使用平面而不是真正的 3D 网格，但是您可以忘记更逼真的效果质量，例如照明、阴影、交叉点……如果您假设物体直立，那么您可以使用房间指标来获取与相机的距离.见:

   • 不同投影的选择标准
   • 估计所拍摄事物的度量

   对于纯 2D 输入，您还可以使用照明来估计 3D 网格，请参阅:

   • 使用代码将任何 2D 图像变成可 3D 打印的雕塑

3. 渲染

   只需将场景渲染回某些图像/视频/屏幕...添加/删除功能.如果您没有过多地改变光照条件，您还可以使用原始图像并直接对其进行渲染.阴影可以通过使像素变暗来实现......为了获得更好的结果，照明/阴影/斑点/等.通常从原始图像中过滤掉，然后直接通过渲染添加.见

   • 白平衡(色彩抑制)公式?
   • 增强动态范围和标准化照明

   渲染过程本身也是平台相关的(除非您通过内存中的低级图形来完成).您可以使用 GDI、DX、OpenGL... 之类的东西，请参阅:

   • 图形渲染

   您还需要用于渲染的相机参数，例如:

   • 与消失点和地平线相关的 3D 对象变换

[谷歌/阅读的基本主题]

1. 二维

   • DIP 数字图像处理
   • 图像分割

2. 3D

   • 矢量数学
   • 齐次坐标
   • 3D 场景重建
   • 3D 图形
   • 正常阴影

3. 平台依赖

   • 图像采集
   • 渲染

1. What algorithms are used for augmented reality like zookazam ?

   I think it analyze image and find planes by contrast, but i don't know how.

2. What topics should I read before starting with app like this?

解决方案

[Prologue]

This is extremly broad topic and mostly off topic in it's current state. I reedited your question but to make your question answerable within the rules/possibilities of this site

You should specify more closely what your augmented reality:

1. should do

   • adding 2D/3D objects with known mesh ...
   • changing light conditions
   • adding/removing body parts/clothes/hairs ...

   a good idea is to provide some example image (sketch) of input/output of what you want to achieve.

2. what input it has

   • video,static image, 2D,stereo,3D. For pure 2D input specify what conditions/markers/illumination/LASER patterns you have to help the reconstruction.
   • what will be in the input image? empty room, persons, specific objects etc.

3. specify target platform

   many algorithms are limited to memory size/bandwidth, CPU power, special HW capabilities etc so it is a good idea to add tag for your platform. The OS and language is also a good idea to add.

[How augmented reality works]

1. acquire input image

   if you are connecting to some device like camera you need to use its driver/framework or something to obtain the image or use some common API it supports. This task is OS dependent. My favorite way on Windows is to use VFW (video for windows) API.

   I would start with some static file(s) from start instead to ease up the debug and incremental building process. (you do not need to wait for camera and stuff to happen on each build). And when your App is ready for live video then switch back to camera...

2. reconstruct the scene into 3D mesh

   if you use 3D cameras like Kinect then this step is not necessary. Otherwise you need to distinguish the object by some segmentation process usually based on the edge detections or color homogenity.

   The quality of the 3D mesh depends on what you want to achieve and what is your input. For example if you want realistic shadows and lighting then you need very good mesh. If the camera is fixed in some room you can predefine the mesh manually (hard code it) and compute just the objects in view. Also the objects detection/segmentation can be done very simply by substracting the empty room image from current view image so the pixels with big difference are the objects.

   you can also use planes instead of real 3D mesh as you suggested in the OP but then you can forget about more realistic quality of effects like lighting,shadows,intersections... if you assume the objects are standing straight then you can use room metrics to obtain the distance from camera. see:

   • selection criteria for different projections
   • estimate measure of photographed things

   For pure 2D input you can also use the illumination to estimate the 3D mesh see:

   • Turn any 2D image into 3D printable sculpture with code

3. render

   Just render the scene back to some image/video/screen... with added/removed features. If you are not changing the light conditions too much you can also use the original image and render directly to it. Shadows can be achieved by darkening the pixels ... For better results with this the illumination/shadows/spots/etc. are usually filtered out from the original image and then added directly by rendering instead. see

   • White balance (Color Suppression) Formula?
   • Enhancing dynamic range and normalizing illumination

   The rendering process itself is also platform dependent (unless you are doing it by low level graphics in memory). You can use things like GDI,DX,OpenGL,... see:

   • Graphics rendering

   You also need camera parameters for rendering like:

   • Transformation of 3D objects related to vanishing points and horizon line

[Basic topics to google/read]

1. 2D

   • DIP digital image processing
   • Image Segmentation

2. 3D

   • Vector math
   • Homogenous coordinates
   • 3D scene reconstruction
   • 3D graphics
   • normal shading

3. paltform dependent

   • image acquisition
   • rendering

这篇关于像zookazam一样增强现实的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！