从捕获的视频帧创建openGL纹理以覆盖视频上的openGL视图的替代方法？ （苹果手机） [英] Alternatives to creating an openGL texture from a captured video frame to overlay an openGL view over video? (iPhone)

问题描述

这主要与增强现实类型应用相关。 Apple提供了有关如何使用AVCaptureSession捕获视频帧（并在需要时将其保存为图像）的信息：

This is mostly relevant for augmented reality type applications. Apple provides information on how to capture video frames (and save them as images if need be) with AVCaptureSession here:

http://developer.apple.com/library/ios/#qa/qa2010/qa1702.html

我知道可以在捕获的视频帧中创建一个openGL纹理，然后在openGL视图中将其用作覆盖其他图形的背景。

I know that it is possible to create an openGL texture out a captured video frame and then use that as a background in the openGL view over which to overlay other graphics.

我想知道这种方法是否有其他选择？上面提到的方法可能是最好的（我不知道是不是），但如果有其他方法可以尝试，那将是很好的。例如，有没有办法直接在AVCaptureVideoPreviewLayer上覆盖openGL视图？

I am wondering if there are any alternatives to this method? The method mentioned above may be the best (I don't know if it is) but if there are alternatives to try it would be good to know. For example, is there a way to overlay the openGL view directly over the AVCaptureVideoPreviewLayer?

推荐答案

你确实可以将OpenGL内容叠加到某个东西上像AVCaptureVideoPreviewLayer，但你的表现会受到影响。 Apple强烈建议您不要将非不透明的OpenGL ES内容叠加在其他显示元素之上。来自适用于iOS的OpenGL ES编程指南：

You can indeed layer OpenGL content over something like AVCaptureVideoPreviewLayer, but your performance will suffer. Apple highly recommends that you not overlay non-opaque OpenGL ES content on top of other display elements. From the OpenGL ES Programming Guide for iOS:

为获得绝对最佳性能，
您的应用程序应仅依靠
OpenGL ES来呈现您的内容。要
执行此操作，请将包含
CAEAGLLayer对象的视图调整大小以匹配
屏幕，将其opaque属性设置为
YES，并确保没有其他Core
动画图层或视图是可见的。
如果您的OpenGL ES图层在其他图层之上合成
，则会使
CAEAGLLayer对象不透明，但
不会消除性能
成本。

For the absolute best performance, your application should rely solely on OpenGL ES to render your content. To do this, size the view that holds your CAEAGLLayer object to match the screen, set its opaque property to YES, and ensure that no other Core Animation layers or views are visible. If your OpenGL ES layer is composited on top of other layers, making your CAEAGLLayer object opaque reduces but doesn’t eliminate the performance cost.

如果您的CAEAGLLayer对象在
图层层次结构中位于其下方图层之上的
混合，则渲染缓冲区的
颜色数据必须位于经过预测，
alpha格式由Core Animation正确合成
。在其他
内容之上混合
OpenGL ES内容会产生严重的性能
罚款。

If your CAEAGLLayer object is blended on top of layers underneath it in the layer hierarchy, the renderbuffer’s color data must be in a premultiplied alpha format to be composited correctly by Core Animation. Blending OpenGL ES content on top of other content has a severe performance penalty.

老实说，将视频作为纹理拉入并将其显示为3-D叠加层背后的广告牌并不难。我的示例应用程序此处将相机视频传输到OpenGL ES（2.0） ）纹理显示在屏幕上。只需进行一些修改，您就可以放置3-D内容。这将比尝试在AVCaptureVideoPreviewLayer上绘制非不透明的3-D内容提供更好的性能。

Honestly, it really isn't that hard to pull in the video as a texture and then display that as a billboard behind your 3-D overlay. My sample application here does passthrough of camera video to an OpenGL ES (2.0) texture for display to the screen. With only a few modifications, you could place 3-D content on top of that. This will give you far better performance than trying to draw non-opaque 3-D content on top of an AVCaptureVideoPreviewLayer.

但是，如果您只是想显示简单的静态关于OpenGL ES内容的UIViews，可以在没有太多性能损失的情况下完成（根据我的经验，帧速率降低约5％）。

However, if you are just wanting to display simple static UIViews over OpenGL ES content, that can be done without much of a performance penalty (~5% reduction in framerate in my experience).

这篇关于从捕获的视频帧创建openGL纹理以覆盖视频上的openGL视图的替代方法？ （苹果手机）的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！