如何使用 iPhone 的相机跟踪运动? [英] How can you track motion using the iPhone's camera?

问题描述

我看到有人开发了一个应用程序，可以使用相机跟踪您的脚，这样您就可以在 iPhone 屏幕上踢虚拟足球.

I saw that someone has made an app that tracks your feet using the camera, so that you can kick a virtual football on your iPhone screen.

你怎么能做这样的事情?有谁知道有关使用 iPhone 摄像头检测和跟踪对象的任何代码示例或其他信息?

How could you do something like this? Does anyone know of any code examples or other information about using the iPhone camera for detecting objects and tracking them?

推荐答案

我刚刚在 SecondConf 上做了一个演讲，我演示了如何使用 iPhone 的摄像头使用 OpenGL ES 2.0 着色器跟踪彩色对象.可以找到该演讲随附的帖子，包括我的幻灯片和所有演示的示例代码 这里.

I just gave a talk at SecondConf where I demonstrated the use of the iPhone's camera to track a colored object using OpenGL ES 2.0 shaders. The post accompanying that talk, including my slides and sample code for all demos can be found here.

我编写的示例应用程序，其代码可以从这里下载，基于 Apple 制作的用于在 WWDC 2007 上展示 Core Image 的示例.该示例在 GPU Gems 3 书的第 27 章.

The sample application I wrote, whose code can be downloaded from here, is based on an example produced by Apple for demonstrating Core Image at WWDC 2007. That example is described in Chapter 27 of the GPU Gems 3 book.

基本思想是，您可以使用自定义 GLSL 着色器实时处理来自 iPhone 相机的图像，确定哪些像素与给定阈值内的目标颜色匹配.然后，这些像素将标准化的 X、Y 坐标嵌入到它们的红色和绿色分量中，而所有其他像素都标记为黑色.然后对整个帧的颜色进行平均以获得彩色对象的质心，您可以在它在相机视图中移动时对其进行跟踪.

The basic idea is that you can use custom GLSL shaders to process images from the iPhone camera in realtime, determining which pixels match a target color within a given threshold. Those pixels then have their normalized X,Y coordinates embedded in their red and green color components, while all other pixels are marked as black. The color of the whole frame is then averaged to obtain the centroid of the colored object, which you can track as it moves across the view of the camera.

虽然这不能解决跟踪更复杂对象(如脚)的情况，但应该能够编写出可以挑选出此类移动对象的着色器.

While this doesn't address the case of tracking a more complex object like a foot, shaders like this should be able to be written that could pick out such a moving object.

作为对上述内容的更新，在我写这篇文章的两年里，我现在开发了一个开源框架 封装了图像和视频的 OpenGL ES 2.0 着色器处理.最近添加的其中一个是 GPUImageMotionDetector 类，它处理场景并检测其中的任何类型的运动.它会将检测到的整体运动的质心和强度返回给您，作为简单回调块的一部分.使用这个框架来做到这一点应该比滚动你自己的解决方案容易得多.

As an update to the above, in the two years since I wrote this I've now developed an open source framework that encapsulates OpenGL ES 2.0 shader processing of images and video. One of the recent additions to that is a GPUImageMotionDetector class that processes a scene and detects any kind of motion within it. It will give you back the centroid and intensity of the overall motion it detects as part of a simple callback block. Using this framework to do this should be a lot easier than rolling your own solution.

这篇关于如何使用 iPhone 的相机跟踪运动?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！