解码器的输出表面如何传递到编码器的输入表面? [英] How does an output surface of a Decoder is passed to an input surface of an Encoder?
问题描述
我试图了解MediaCodec的表面对表面方法是如何工作的.在仅ByteBuffer方法中，将解码后的数据放置在OutputBuffers中.可以手动处理这些未编码的数据，然后将其传递到编码器的InputBuffers.
I'm trying to understand how the surface-to-surface approach works with MediaCodec. In a ByteBuffer only approach, decoded data is placed in OutputBuffers. This non-encoded data can be processed manually then passed to the InputBuffers of an Encoder.
If we give a look at an example from Android MediaCodec CTS using a surface to surface approach to pass data between a decoder and an encoder, we configure the Decoder to output the decoded data onto a Surface called outputSurface, and we configure the Encoder to receive the data on a Surface called inputSurface.
在文档中，createInputSurface以及此表面在编码器配置中的用法描述如下:
In the documentation, the createInputSurface and the usage of this surface in the configuration of the Encoder is described as so:
createInputSurface():请求Surface代替输入缓冲区用作编码器的输入.
createInputSurface(): Requests a Surface to use as the input to an encoder, in place of input buffers.
换句话说，这在ByteBuffers声明的CTS示例中可见:仅编码器没有InputBuffers.你有:
In other terms, and this is visible in the CTS example in the ByteBuffers declarations: there is just no InputBuffers for the Encoder. You have:
- DecoderInputBuffers(从MediaExtractor接收视频轨道样本)
- DecoderOutputBuffers(输出以拉出已解码的yuv帧)
- 什么都没有. (嗯...输入Surface.)
- EncoderOutputBuffers(用于输出重新编码的内容以传递到多路复用器的输出)
您可以使用以下代码行，而不是在Encoder InputBuffers中排队数据:
Instead of enqueu-ing data in the Encoder InputBuffers, you have these line of codes:
outputSurface.awaitNewImage();
outputSurface.drawImage();
inputSurface.setPresentationTime(videoDecoderOutputBufferInfo.presentationTimeUs * 1000);
inputSurface.swapBuffers();
解码器的ouputSurface内容如何传递到编码器的inputSurface?幕后究竟发生了什么?
How is the ouputSurface content of the Decoder passed to the inputSurface of the Encoder? What is concretely happening behind the curtain?
推荐答案
解码器/编码器的输出/输入Surface分别是特殊配置(物理上连续的或保留的等)内存，专用硬件(例如，GPUs，硬件(加速)编解码器)或软件模块可以以最适合性能需求的方式使用(通过使用诸如硬件加速，DMA等).
The decoder's/encoder's output/input Surface respectively is a specially configured (either physically contiguous or reserved etc) piece of memory which specialised hardwares (for example, GPUs, hardware (accelerated) codecs) or software modules can use in a fashion best suited for performance needs (by using features such as hardware acceleration, DMA etc).
更具体地说，例如，在当前上下文中，解码器的输出Surface由SurfaceTexture支持，因此可以在OpenGL environment中使用，用作外部纹理，编码器可以在Surface上渲染之前进行的任何类型的处理，编码器都可以从中进行读取和编码，以创建最终的视频帧.
More specifically, in the current context for instance, the decoder's output Surface is backed by SurfaceTexture, so that it can be used in an OpenGL environment to be used as an external texture for any kind of processing before it is rendered on the Surface from which the encoder can read and encode to create the final video frame.
并非巧合的是，OpenGL只能渲染成这样的Surface.
Not coincidentally, OpenGL can only render to such a Surface.
因此，解码器充当原始视频帧的提供者，表面(纹理)是载体 ， OpenGL medium 将其呈现到编码器的 input Surface (即目标 >(用于(要编码的)视频帧.
So the decoder acts as the provider of raw video frame, the Surface (Texture) the carrier, OpenGL the medium to render it to the Encoder's input Surface which is the destination for the (to be encoded) video frame.
要进一步满足您的好奇心，请检查使用MediaCodec编辑帧和编码有关更多详细信息.
To further satiate your curiosity, check Editing frames and encoding with MediaCodec for more details.
您可以在grafika中检查子项目.连续摄像机或显示+捕获摄像头，当前可将摄像头帧(馈送到SurfaceTexture)渲染为视频(并显示).因此从本质上讲，唯一的变化是MediaCodec向SurfaceTexture馈送帧，而不是向相机馈送帧.
You can check subprojects in grafika Continuous Camera or Show + capture camera, which currently renders Camera frames (fed to SurfaceTexture) to a Video (and display). So essentially, the only change is the MediaCodec feeding frames to SurfaceTexture instead of the Camera.
Google CTS DecodeEditEncodeTest does exactly the same and can be used as a reference in order to make the learning curve smoother.
正如fadden指出的，从最基础的内容开始，请使用 Android图形教程
To start from the very basics, as fadden pointed out use Android graphics tutorials
这篇关于解码器的输出表面如何传递到编码器的输入表面?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
