Android的媒体codeC德codeR输入/输出帧计数 [英] Android MediaCodec decoder input/output frame count
问题描述
我工作的视频转码在Android中，并使用标准方法这些样品解压/解codeA视频。我测试相同的过程在不同的视频设备不同的设备，而我发现去codeR输入/输出的帧计数有问题。
I'm working on video transcoding in Android, and using the standard method as these samples to extract/decode a video. I test the same process on different devices with different video devices, and I found a problem on the frame count of decoder input/output.
有关一段时间code问题在<一个href=\"http://stackoverflow.com/questions/22368611/inconsistent-sample-time-$p$psentation-time-during-video-decoding\">this问题，我用一个队列来记录视频中提取样本，检查队列时，我得到了去codeR帧输出，像下面的codeS：
（我省略了编码相关的codeS使其更清晰）
For some timecode issues as in this question, I use a queue to record the extracted video samples, and check the queue when I got a decoder frame output, like the following codes: (I omit the encoding-related codes to make it clearer)
Queue<Long> sample_time_queue = new LinkedList<Long>();
....
// in transcoding loop
if (is_decode_input_done == false)
{
int decode_input_index = decoder.dequeueInputBuffer(TIMEOUT_USEC);
if (decode_input_index >= 0)
{
ByteBuffer decoder_input_buffer = decode_input_buffers[decode_input_index];
int sample_size = extractor.readSampleData(decoder_input_buffer, 0);
if (sample_size < 0)
{
decoder.queueInputBuffer(decode_input_index, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
is_decode_input_done = true;
}
else
{
long sample_time = extractor.getSampleTime();
decoder.queueInputBuffer(decode_input_index, 0, sample_size, sample_time, 0);
sample_time_queue.offer(sample_time);
extractor.advance();
}
}
else
{
DumpLog(TAG, "Decoder dequeueInputBuffer timed out! Try again later");
}
}
....
if (is_decode_output_done == false)
{
int decode_output_index = decoder.dequeueOutputBuffer(decode_buffer_info, TIMEOUT_USEC);
switch (decode_output_index)
{
case MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED:
{
....
break;
}
case MediaCodec.INFO_OUTPUT_FORMAT_CHANGED:
{
....
break;
}
case MediaCodec.INFO_TRY_AGAIN_LATER:
{
DumpLog(TAG, "Decoder dequeueOutputBuffer timed out! Try again later");
break;
}
default:
{
ByteBuffer decode_output_buffer = decode_output_buffers[decode_output_index];
long ptime_us = decode_buffer_info.presentationTimeUs;
boolean is_decode_EOS = ((decode_buffer_info.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0);
if (is_decode_EOS)
{
// Decoder gives an EOS output.
is_decode_output_done = true;
....
}
else
{
// The frame time may not be consistent for some videos.
// As a workaround, we use a frame time queue to guard this.
long sample_time = sample_time_queue.poll();
if (sample_time == ptime_us)
{
// Very good, the decoder input/output time is consistent.
}
else
{
// If the decoder input/output frame count is consistent, we can trust the sample time.
ptime_us = sample_time;
}
// process this frame
....
}
decoder.releaseOutputBuffer(decode_output_index, false);
}
}
}
在一些情况下，队列可以正确的PTS若德codeR给出误差值（例如大量的0的）。然而，仍有约德codeR输入/输出的帧计数的一些问题。
In some cases, the queue can "correct" the PTS if the decoder gives error value (e.g. a lot of 0s). However, there are still some issues about the frame count of decoder input/output.
在是HTC One 801E设备，我用codeC OMX.qcom.video.de coder.avc脱code的视频（使用MIME类型的视频/ AVC）。采样时间和PTS匹配良好的帧，除了最后一个。
例如，如果抽取馈送100帧，然后EOS到去codeR中，第一99日codeD帧具有完全相同的时间的值，但在最后的帧丢失和I从解得到输出的EOS $ C $铬。我通过测试EN codeD不同视频内置摄像头，通过ffmpeg的合并器，或通过在Windows中的视频处理的AP。他们都有最后一帧消失了。
On an HTC One 801e device, I use the codec OMX.qcom.video.decoder.avc to decode the video (with MIME types video/avc). The sample time and PTS is matched well for the frames, except the last one. For example, if the extractor feeds 100 frames and then EOS to the decoder, the first 99 decoded frames has the exactly same time values, but the last frame is missing and I get output EOS from the decoder. I test different videos encoded by the built-in camera, by ffmpeg muxer, or by a video processing AP on Windows. All of them have the last one frame disappeared.
在一些胶垫OMX.MTK.VIDEO.DE codeR.AVC codeC，事情变得更加混乱。有些视频已经从去codeR和输入/输出帧计数良好的PTS是正确的（即当解码完成队列为空）。有些视频已经与德codeR输出坏PTS（我仍然可以通过队列纠正）一致的输入/输出帧计数。对于一些视频，很多帧被解码期间失踪。例如，提取器得到7第二视频210帧，但脱codeR仅输出的最后180帧。这是不可能使用相同的解决方法来恢复PTS。
On some pads with OMX.MTK.VIDEO.DECODER.AVC codec, things becomes more confused. Some videos has good PTS from the decoder and the input/output frame count is correct (i.e. the queue is empty when the decoding is done.). Some videos has consistent input/output frame count with bad PTS in decoder output (and I can still correct them by the queue). For some videos, a lot of frames are missing during the decoding. For example, the extractor get 210 frames in a 7 second video, but the decoder only output the last 180 frames. It is impossible to recover the PTS using the same workaround.
有没有什么办法可以期待一个媒体codeC德codeR输入/输出帧计数？或更准确地，知道哪些帧（s）的德codeR被丢弃，而提取与正确采样时间给它的视频样本
Is there any way to expect the input/output frame count for a MediaCodec decoder? Or more accurately, to know which frame(s) are dropped by the decoder while the extractor gives it video samples with correct sample time?
推荐答案
同样的基本故事在<一个href=\"http://stackoverflow.com/questions/22368611/inconsistent-sample-time-$p$psentation-time-during-video-decoding\">the另一个问题。 pre-4.3，没有测试，确认每一帧送入一间codeR或去codeR出来的另一面。我记得一些设备能可靠滴在某些测试的最后一帧，直到codeCS被固定在4.3。
Same basic story as in the other question. Pre-4.3, there were no tests confirming that every frame fed to an encoder or decoder came out the other side. I recall that some devices would reliably drop the last frame in certain tests until the codecs were fixed in 4.3.
我没有寻找当时的解决办法，所以我不知道如果存在的话。延迟发送EOS可能有帮助，如果它导致一些早期关闭之前。
I didn't search for a workaround at the time, so I don't know if one exists. Delaying before sending EOS might help if it's causing something to shut down early.
我不相信我所见到的设备落大量帧。这似乎是一个不寻常的情况下，它会在以类似的方式行使媒体codeC 即使没有仔细测试所有应用有明显的。
I don't believe I ever saw a device drop large numbers of frames. This seems like an unusual case, as it would have been noticeable in any apps that exercised MediaCodec in similar ways even without careful testing.
这篇关于Android的媒体codeC德codeR输入/输出帧计数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
