从捕获的视频帧创建openGL纹理以覆盖视频上的openGL视图的替代方法?(iPhone) [英] Alternatives to creating an openGL texture from a captured video frame to overlay an openGL view over video? (iPhone)

问题描述

这主要与增强现实类型的应用程序相关.Apple 在此处提供了有关如何使用 AVCaptureSession 捕获视频帧(并在需要时将其另存为图像)的信息:

This is mostly relevant for augmented reality type applications. Apple provides information on how to capture video frames (and save them as images if need be) with AVCaptureSession here:

http://developer.apple.com/library/ios/#qa/qa2010/qa1702.html

我知道可以从捕获的视频帧中创建一个 openGL 纹理，然后将其用作 openGL 视图中的背景，以覆盖其他图形.

I know that it is possible to create an openGL texture out a captured video frame and then use that as a background in the openGL view over which to overlay other graphics.

我想知道这种方法是否有任何替代方法?上面提到的方法可能是最好的(我不知道是不是)，但如果有其他方法可以尝试，最好知道.例如，有没有办法将 openGL 视图直接覆盖在 AVCaptureVideoPreviewLayer 上?

I am wondering if there are any alternatives to this method? The method mentioned above may be the best (I don't know if it is) but if there are alternatives to try it would be good to know. For example, is there a way to overlay the openGL view directly over the AVCaptureVideoPreviewLayer?

推荐答案

您确实可以在 AVCaptureVideoPreviewLayer 之类的东西上分层 OpenGL 内容，但您的性能会受到影响.Apple 强烈建议您不要将不透明的 OpenGL ES 内容覆盖在其他显示元素之上.来自 OpenGL ES iOS 编程指南:

You can indeed layer OpenGL content over something like AVCaptureVideoPreviewLayer, but your performance will suffer. Apple highly recommends that you not overlay non-opaque OpenGL ES content on top of other display elements. From the OpenGL ES Programming Guide for iOS:

为了获得绝对的最佳性能，您的应用程序应仅依赖于OpenGL ES 来渲染你的内容.到这样做，调整容纳你的视图的大小CAEAGLLayer 对象来匹配屏幕，将其 opaque 属性设置为是，并确保没有其他核心动画层或视图是可见的.如果您的 OpenGL ES 层是合成的在其他层之上，使您的CAEAGLLayer 对象不透明减少但不会消除性能成本.

For the absolute best performance, your application should rely solely on OpenGL ES to render your content. To do this, size the view that holds your CAEAGLLayer object to match the screen, set its opaque property to YES, and ensure that no other Core Animation layers or views are visible. If your OpenGL ES layer is composited on top of other layers, making your CAEAGLLayer object opaque reduces but doesn’t eliminate the performance cost.

如果您的 CAEAGLLayer 对象是混合的在它下面的层之上层层次结构，渲染缓冲区的颜色数据必须在预乘要合成的 alpha 格式由核心动​​画正确.混合OpenGL ES 内容在其他之上内容表现严重罚款.

If your CAEAGLLayer object is blended on top of layers underneath it in the layer hierarchy, the renderbuffer’s color data must be in a premultiplied alpha format to be composited correctly by Core Animation. Blending OpenGL ES content on top of other content has a severe performance penalty.

老实说，将视频作为纹理拉入然后将其显示为 3-D 叠加层后面的广告牌并不难.我的示例应用程序 here 将相机视频传递到 OpenGL ES (2.0) 纹理显示到屏幕上.只需进行一些修改，您就可以在其上放置 3-D 内容.与尝试在 AVCaptureVideoPreviewLayer 之上绘制非透明 3-D 内容相比，这将为您提供更好的性能.

Honestly, it really isn't that hard to pull in the video as a texture and then display that as a billboard behind your 3-D overlay. My sample application here does passthrough of camera video to an OpenGL ES (2.0) texture for display to the screen. With only a few modifications, you could place 3-D content on top of that. This will give you far better performance than trying to draw non-opaque 3-D content on top of an AVCaptureVideoPreviewLayer.

但是，如果您只是想在 OpenGL ES 内容上显示简单的静态 UIView，则可以在没有太多性能损失的情况下完成(根据我的经验，帧速率降低了约 5%).

However, if you are just wanting to display simple static UIViews over OpenGL ES content, that can be done without much of a performance penalty (~5% reduction in framerate in my experience).

这篇关于从捕获的视频帧创建openGL纹理以覆盖视频上的openGL视图的替代方法?(iPhone)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！