如何减少在媒体codeC视频/ AVC解码延时 [英] How to reduce latency in MediaCodec video/avc decoding

问题描述

我执行的 MoviePlayer。 java的在 Grafika 媒体$ C $毫升样品code上的Nexus 5运行我把日志语句在这些地方：

I performed some simple timing of MoviePlayer.java in the Grafika MediaCodec sample code running on a Nexus 5. I put a log statement at these locations:

目前管线203之前

decoder.queueInputBuffer

目前行244后

decoder.dequeueOutputBuffer

我用相关的日志语句 presentationTimeUs 。

下面是从logcat的摘录：

Here is an excerpt from logcat:

01-29 10:56:43.295: I/Grafika(21286): queueInputBuffer index/pts, 2,0
01-29 10:56:43.305: I/Grafika(21286): queueInputBuffer index/pts, 0,33100
01-29 10:56:43.315: I/Grafika(21286): queueInputBuffer index/pts, 3,66466
01-29 10:56:43.325: I/Grafika(21286): queueInputBuffer index/pts, 1,99833
01-29 10:56:43.325: I/Grafika(21286): queueInputBuffer index/pts, 2,133200
01-29 10:56:43.335: I/Grafika(21286): queueInputBuffer index/pts, 0,166566
01-29 10:56:43.345: I/ATSParser(21286): discontinuity on stream pid 0x1011
01-29 10:56:43.345: I/ATSParser(21286): discontinuity on stream pid 0x1100
01-29 10:56:43.345: I/Grafika(21286): queueInputBuffer index/pts, 3,199933
01-29 10:56:43.345: I/Grafika(21286): dequeueOutputBuffer index/pts, 7,0
01-29 10:56:43.345: I/Grafika(21286): queueInputBuffer index/pts, 1,300033
01-29 10:56:43.355: I/Grafika(21286): dequeueOutputBuffer index/pts, 6,33100
01-29 10:56:43.385: I/Grafika(21286): queueInputBuffer index/pts, 2,333400
01-29 10:56:43.385: I/Grafika(21286): dequeueOutputBuffer index/pts, 5,66466
01-29 10:56:43.415: I/Grafika(21286): queueInputBuffer index/pts, 0,366766
01-29 10:56:43.415: I/Grafika(21286): dequeueOutputBuffer index/pts, 4,99833
01-29 10:56:43.445: I/Grafika(21286): queueInputBuffer index/pts, 3,400133
01-29 10:56:43.445: I/Grafika(21286): dequeueOutputBuffer index/pts, 3,133200

我发现从当第一输入缓冲器排队时相应的输出缓冲器出列是50米利斯的时间差。这似乎是一个大量的时间进行硬件加速解码。

I found the time difference from when the first input buffer is queued to when the corresponding output buffer is dequeued is 50 millis. This seems like a lot of time for HW accelerated decoding.

有没有办法减少这种延迟？

推荐答案

我想你看到一些效果独特的第一帧。我反复实验，以进一步增加迫使 doRender = FALSE 周围行244，以避免用于管理输出帧速率睡眠电话。我明白了：

I think you're seeing some effects unique to the first frame. I repeated your experiment, with the further addition of forcing doRender = false around line 244 to avoid the sleep calls used to manage the output frame rate. I see:

01-29 14:05:36.552  9115  9224 I Grafika : queueInputBuffer index/pts, 2,0
01-29 14:05:36.562  9115  9224 I Grafika : queueInputBuffer index/pts, 0,66655
01-29 14:05:36.572  9115  9224 I Grafika : queueInputBuffer index/pts, 3,133288
01-29 14:05:36.582  9115  9224 I Grafika : queueInputBuffer index/pts, 1,199955

01-29 14:05:36.602  9115  9224 I Grafika : dequeueOutputBuffer index/pts, 4,0
01-29 14:05:36.602  9115  9224 I Grafika : dequeueOutputBuffer index/pts, 3,66655
01-29 14:05:36.602  9115  9224 I Grafika : dequeueOutputBuffer index/pts, 2,133288
01-29 14:05:36.612  9115  9224 I Grafika : dequeueOutputBuffer index/pts, 4,199955

（为了清楚起见去多余行）。这证实了你的结果。需要注意的是，虽然有输入和输出点= 0，随后输出帧​​是可几乎立即之间一个50ms的滞后。我使用的视频是相机test.mp4（720P摄像机输出）。

(Extraneous lines removed for clarity.) This confirms your results. Note that, while there was a 50ms lag between input and output for pts=0, the subsequent output frames were available almost instantly. The video I used was "camera-test.mp4" (720p camera output).

有关深入了解为什么发生这种情况，看看其他的东西在日志中，并在那里出现。从第一个 queueInputBuffer 日志行，开始计数出现了和第一 dequeueOutputBuffer 线之间的日志数量。我算60行输出的OMX-VDEC-1080P在我的。现在算上很多OMX-VDEC线的显示后输出缓冲器开始出现。我看没有，直到视频结束。

For insight into why this is happening, take a look at the other stuff in the log, and where it appears. Starting from the first queueInputBuffer log line, count the number of logs that appear between that and the first dequeueOutputBuffer line. I count about 60 lines of output from OMX-VDEC-1080P on mine. Now count how many OMX-VDEC lines appear after output buffers start appearing. I see none until the video ends.

视频去codeR显然推迟了一些昂贵的初始化，直到数据可用。因此，接下来的问题是...有多少数据是否需要？我加了一个500毫秒的睡眠提交第二帧（PTS == 66633）之后。其结果是：提交的两帧，500毫秒的停顿，提交了两个框架，一大堆OMX-VDEC日志。如此看来，去codeR想几帧的时候，才开始。

The video decoder is clearly deferring some expensive initialization until data is available. So the next question is... how much data does it need? I added a 500ms sleep after submitting the second frame (pts==66633). The result: two frames submitted, 500ms pause, two frames submitted, big pile of OMX-VDEC logs. So it seems that the decoder wants several frames before it will start.

这表明，我们可以通过快速前几帧喂养减少开机等待时间。为了验证这一点，我改变了 TIMEOUT_USEC 为零，所以它会迅速作出反应，但烧CPU。新的日志输出（你的日志，不睡觉，不渲染）：

This suggests that we can reduce the start-up latency by feeding the first few frames in quickly. To test that, I changed TIMEOUT_USEC to zero, so it'll respond quickly but burn CPU. New log output (your logs, no sleep, no rendering):

01-29 14:29:04.542 10560 10599 I Grafika : queueInputBuffer index/pts, 0,0
01-29 14:29:04.542 10560 10599 I Grafika : queueInputBuffer index/pts, 2,66633
01-29 14:29:04.542 10560 10599 I Grafika : queueInputBuffer index/pts, 3,133288
...
01-29 14:29:04.572 10560 10599 I Grafika : dequeueOutputBuffer index/pts, 4,0
01-29 14:29:04.572 10560 10599 I Grafika : dequeueOutputBuffer index/pts, 3,66633
01-29 14:29:04.572 10560 10599 I Grafika : dequeueOutputBuffer index/pts, 2,133288

通过喂养在快速初始框架，我们已经减少了从50毫秒的初始延迟至30ms。

By feeding the initial frames in quickly, we've reduced the initial latency from 50ms to 30ms.

（注意：所有时间戳在如何结束'2'？似乎用于日志时间的计时器被四舍五入到最接近的10毫秒，因此实际时间差可能略有不同。）

(Notice how all the timestamps end in '2'? The timer used for the log times appears to be rounding to the nearest 10ms, so the actual time delta may be slightly different.)

我们正在慢慢地喂初始帧的原因是，我们试图从去codeR后，每输入缓冲器被提交漏极开路输出，等待10毫秒为从未出现的输出。我最初的想法是，我们希望要等到超时上的或者的 dequeueInputBuffer（）的或的 dequeueOutputBuffer（），但不能同时 - 也许使用输入和快速轮询输出超时在第一，然后切换到超时输出，当我们用完输入养活。 （对于这个问题，在初始的超时输入可以是-1，因为我们知道没有什么事情发生，直到第一个输入缓冲区排队。）

The reason we're feeding the initial frames slowly is that we're trying to drain output from the decoder after each input buffer is submitted, waiting 10ms for output that never appears. My initial thought is that we want to want to wait for a timeout on either dequeueInputBuffer() or dequeueOutputBuffer(), but not both -- maybe use a timeout on input and a quick poll for output at first, then switch to a timeout on output when we run out of input to feed. (For that matter, the initial timeout for input could be -1, since we know nothing is going to happen until the first input buffer is queued.)

我不知道是否有一种方法，以进一步减少延迟。

I don't know if there is a way to reduce the latency further.

这篇关于如何减少在媒体codeC视频/ AVC解码延时的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！