Android MediaCodec似乎可以缓冲H264帧 [英] Android MediaCodec appears to buffer H264 frames

问题描述

我正在手动读取RTP/H264流，并将H264帧传递给Android MediaCodec.我使用"markerBit"作为框架的边框. MediaCodec绑定到OpenGL纹理(SurfaceTexture). 总的来说，一切正常.但是解码器似乎可以缓冲帧.如果我将一帧放入解码器，则不会立即渲染到纹理.在解码器中再放置2-3帧后，第一帧将渲染为纹理.

I'm manually reading a RTP/H264 stream and pass the H264 frames to the Android MediaCodec. I use the "markerBit" as a border for the frames. The MediaCodec is tied to a OpenGL Texture (SurfaceTexture). In general everything works fine. But the Decoder appears to buffer frames. If I put a frame in the decoder it is not rendered immediately to the texture. After I put 2-3 frames more in the decoder the first frame is rendered to the texture.

我正在针对Android 4.4.4实施.

I'm implementing against Android 4.4.4.

private static final int INFINITE_TIMEOUT = -1;
private static final int TIMEOUT_OUTPUT_BUFFER_MEDIA_CODEC = 1000;
...
int bufferIndex = codec.dequeueInputBuffer(INFINITE_TIMEOUT);
if (bufferIndex < 0) {
  throw new RuntimeException("Error");
}

ByteBuffer inputBuffer = inputBuffers[bufferIndex];
inputBuffer.clear();

// Copy H264 data to inputBuffer
h264Frame.fill(inputBuffer);

codec.queueInputBuffer(bufferIndex, 0, inputBuffer.position(), 0, 0);
drainOutputBuffers();
...

和

private boolean drainOutputBuffers() {
MediaCodec.BufferInfo buffInfo = new MediaCodec.BufferInfo();

int outputBufferIndex = codec.dequeueOutputBuffer(buffInfo, TIMEOUT_OUTPUT_BUFFER_MEDIA_CODEC);

if (outputBufferIndex >= 0) {
  codec.releaseOutputBuffer(outputBufferIndex, true);
  return true;
}

switch (outputBufferIndex) {
  case MediaCodec.INFO_TRY_AGAIN_LATER:
    LOG.debug("Could not dequeue output buffer. Try again later");
    break;
  case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
    LOG.warn("The output format has changed.");
    break;
  case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
    LOG.warn("The output buffers has changed.");
    break;
  default:
    LOG.warn("The output buffer index was negative: {}", outputBufferIndex);
}
return false;
}

在渲染方面，我使用"onFrameAvailable"回调来检查是否必须更新openGl线程上的纹理.我用于检查的标志由锁(同步)保护.

On the rendering side I use the "onFrameAvailable" callback for checking if I have to update the texture on the openGl Thread. The flag I use for checking is guarded by a lock (synchronized).

我怀疑演示时间戳可能会影响渲染.但是我将其设置为0.因此，我假设应该渲染帧而没有延迟.

I suspect that the presentation timestamp may influence the rendering. But I set it to 0. Thus I assume the frame should be rendered without a delay.

我想将框架渲染为纹理，而不必放置其他框架.

I'd like to have the frame rendered to the texture without having to put additional frames.

推荐答案

来自MediaCodec 文档

From the MediaCodec documentation

执行状态具有三个子状态:刷新"，运行"和运行" 流结束. start()之后，编解码器立即处于刷新状态 子状态，其中包含所有缓冲区.第一次输入 缓冲区出队，编解码器移至运行中"子状态 花费了大部分时间. 将输入缓冲区与 流结束标记，编解码器过渡到End-of-Stream 子状态.在这种状态下，编解码器不再接受进一步的输入 缓冲区，但仍会生成输出缓冲区，直到流结束 在输出上达到.您可以移至以下位置的Flushed子状态 随时使用flush()处于执行状态.

The Executing state has three sub-states: Flushed, Running and End-of-Stream. Immediately after start() the codec is in the Flushed sub-state, where it holds all the buffers. As soon as the first input buffer is dequeued, the codec moves to the Running sub-state, where it spends most of its life. When you queue an input buffer with the end-of-stream marker, the codec transitions to the End-of-Stream sub-state. In this state the codec no longer accepts further input buffers, but still generates output buffers until the end-of-stream is reached on the output. You can move back to the Flushed sub-state at any time while in the Executing state using flush().

您需要使用end-of-stream标记排队输入缓冲区".对您提供给解码器的第一帧执行此操作(确保它是关键帧).

You need to "queue an input buffer with the end-of-stream marker". Do this with the first frame you feed to the decoder (make sure it is a keyframe).

这是告诉解码器不要再等待帧，因此立即开始播放.否则，在看到任何东西之前先送入3或4帧是正常的.这是对所有MPEG解码器的期望，与Android无关.

This point is to tell the decoder not to expect anymore frames and therefore begin playback immediately. Otherwise it's normal to feed 3 or 4 frames before seeing anything. This an expectation of all MPEG decoders and is not Android-related.

这篇关于Android MediaCodec似乎可以缓冲H264帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！