如何处理Android MediaCodec解码器的第一个输出ByteBuffer? [英] How do I handle first output ByteBuffers of Android MediaCodec decoder?

问题描述

我正在尝试使用Android的MediaCodec套件编写音频重采样器.

I am trying to write an audio resampler using Android's MediaCodec suite.

我目前正在将MP3立体声音频文件输入到MediaExtractor中，然后由MediaCodec对其进行解码.源音频的采样率为48000.

I am currently feeding an MP3 stereo audio file into a MediaExtractor which is then decoded by a MediaCodec. The sample rate of the source audio is 48000.

我不理解的是我从解码器接收到的前四个输出缓冲区:

What I don't understand is the first four output buffers I receive from the decoder:

1. 大小0，时间0
2. 大小0，时间24000
3. 大小4312，时间48000
4. 大小4608，时间72000
5. 大小4608，时间96000
6. 等

1. size 0, time 0
2. size 0, time 24000
3. size 4312, time 48000
4. size 4608, time 72000
5. size 4608, time 96000
6. etc.

来自此答案，此答案和本文，我相信前两个缓冲区只是传播编码器延迟"，可能会被丢弃.但是，我列出的第三个缓冲区使我陷入循环.

From this answer, this answer, and this article, I believe the first two buffers are merely propagated "encoder delay" and may just be thrown out. However, the third buffer I have listed throws me for a loop.

对于4号缓冲区(及更高版本)，其数学公式为:

For buffer #4 (and onward), the math works out:

((4608 bytes) / (2 bytes/sample) / (2 channels)) 
    / ((48,000 samples/sec) / (1,000,000 us/sec))
= 24,000 us (i.e. the change in time between buffers)

虽然缓冲区3发生了什么?对数据的直接了解表明，音频在48000 us的时间开始播放，然后在72000 us的标记之前暂时暂停，这时它开始连续播放而没有间断.

What is going on with buffer #3 though? A straightforward take on the data suggests that the audio begins playing at time 48000 us and then pauses momentarily before the 72000 us mark, at which point it begins to play continuously with no breaks.

似乎缓冲区#3的数据之前有296个隐藏的0，但是此偏移量似乎未由我的代码中的任何变量指示.谁能为我阐明一下?

It seems more likely that there are 296 hidden 0's before the data of buffer #3, but this offset doesn't seem to be indicated by any variables in my code. Can anyone shed some light on this for me?

推荐答案

据我所知，音频MediaCodec素材*并不真正在乎与每个缓冲区关联的时间戳.取而代之的是，它假设指定的字节流中没有空洞，而是使用指定的比特率，神奇地重新计算了每个数据的时间戳.

As far as I have figured out, audio MediaCodec stuffs* don't really care what the timestamps are associated with each buffer. Instead, it just magically recalculates what the timestamps should be for each piece of data, using the specified bitrate, by assuming there are no holes in the flow of bytes.

作为该假设的支持证据，此答案中的解决方案之一只是建议增加时间戳值，而不是实际计算出正确的时间戳.

As a supporting piece of evidence to this hypothesis, one of the pieces of the solution in this answer simply suggests incrementing timestamp values and not actually calculating the correct timestamps.

因此在此问题的示例中，音频MediaCodec填充*将完全忽略所有时间戳值. MediaCodec将缓冲区#3字节#1假定为时间0，并根据到目前为止已处理的字节数推断缓冲区#4字节#1的时间，并且 not 设为24000或48000.

So in the example of this question, audio MediaCodec stuffs* would completely ignore all of the timestamp values. Buffer #3 byte #1 would be assumed by MediaCodec to be time 0, and the time for buffer #4 byte #1 would be inferred from the number of bytes processed thus far and not taken as 24000 or 48000.

*即MediaCodec对象或一些相关的自定义组件

*namely a MediaCodec object or some related custom component

注意:MediaCodec视频编码器似乎确实关心时间戳.

Note: MediaCodec video encoder does seem to care about timestamps.

这篇关于如何处理Android MediaCodec解码器的第一个输出ByteBuffer?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！