多路复用附件B MPEG-TS时，是否等同于SPS和PPS的音频?什么是"DecoderInfo"? [英] Audio equivalent of SPS and PPS when muxing Annex B MPEG-TS? What is "DecoderInfo"?

问题描述

我正在使用 Bento4 库将附件B TS(MPEG-2传输流)文件与我分别从VideoToolbox和AVFoundation生成的h264视频和AAC音频流，作为HLS(HTTP实时流)流的源数据.这个问题不一定是Bento4特有的:我试图理解底层概念，以便我可以完成任务，最好使用Apple libs.

I'm using the Bento4 library to mux an Annex B TS (MPEG-2 transport stream) file with my h264 video and AAC audio streams that are being generated from VideoToolbox and AVFoundation respectively, as source data for a HLS (HTTP Live Streaming) stream. This question is not necessarily Bento4-specific: I'm trying to understand the underlying concepts so that I can accomplish the task, preferably by using Apple libs.

到目前为止，我已经弄清楚了如何通过从我的 CMVideoFormatDescriptionRef 中获取各种数据，以及最重要的是通过生成SPS和PPS来创建 AP4_AvcSampleDescription 分别使用 CMVideoFormatDescriptionGetH264ParameterSetAtIndex 的索引0和1，我可以将它们作为字节缓冲区粘贴到Bento4中.太好了，这就是我需要的所有标头信息，以便我可以要求Bento4将视频多路复用到ts文件中！

So far, I've figured out how to create an AP4_AvcSampleDescription by getting various kinds of data out of my CMVideoFormatDescriptionRef, and most importantly by generating an SPS and PPS using index 0 and 1 respectively of CMVideoFormatDescriptionGetH264ParameterSetAtIndex that I can just stick as byte buffers into Bento4. Great, that's all the header information I need so that I can ask Bento4 to mux video into a ts file!

现在，我正在尝试将音频多路复用到同一文件中.我正在使用我的 CMAudioFormatDescriptionRef 获得构建我的 AP4_MpegAudioSampleDescription 所需的信息，Bento4使用该信息来制作必要的QT原子和标头.但是，如果这些字段是解码器信息"字节缓冲区，则没有说明它是什么，或者是从数据生成一个的代码.我本来希望有一个 CMAudioFormatDescriptionGetDecoderInfo 之类的东西，但是我找不到类似的东西.苹果库中是否有这样的功能?还是有一个关于如何生成这些数据的好的规范?

Now I'm trying to mux audio into the same file. I'm using my CMAudioFormatDescriptionRef to get the required information to construct my AP4_MpegAudioSampleDescription, which Bento4 uses to make the necessary QT atoms and headers. However, one if the fields is a "decoder info" byte buffer, with no explanation of what it is, or code to generate one from data. I would have hoped to have a CMAudioFormatDescriptionGetDecoderInfo or something, but I can't find anything like that. Is there such a function in any Apple library? Or is there a nice spec that I haven't found on how to generate this data?

或者，我是走错了路吗?有没有更简单的方法可以从Mac/iOS代码库中混合ts文件?

Or alternatively, am I walking down the wrong path? Is there an easier way to mux ts files from a Mac/iOS code base?

推荐答案

将音频混合到MPEG-TS中非常容易，并且不需要像视频流那样复杂的标题！在将每个示例缓冲区写为PES之前，只需要一个7字节的ADTS标头即可.

Muxing audio into an MPEG-TS is surprisingly easy, and does not require a complex header like a video stream does! It only requires a 7-byte ADTS header before each sample buffer, before you write it as a PES.

Bento4仅使用"DecoderInfo"缓冲区将其解析为 AP4_Mp4AudioDecoderConfig 实例，以便它可以提取ADTS标头所需的信息.我没有在获取数据时如此费力，而是复制粘贴了 AP4_Mpeg2TsAudioSampleStream :: WriteSample ，编写了 CMSampleBufferRef .可以很容易地将其通用化为其他音频框架，但是我将其按原样粘贴以供参考:

Bento4 only uses the "DecoderInfo" buffer in order to parse it into an AP4_Mp4AudioDecoderConfig instance, so that it can extract the information needed for the ADTS header. Instead of being so roundabout in acquiring this data, I made a copy-paste of AP4_Mpeg2TsAudioSampleStream::WriteSample that writes a CMSampleBufferRef. It can easily be generalized for other audio frameworks, but I'll just paste it as-is here for reference:

// These two functions are copy-pasted from Ap4Mpeg2Ts.cpp
static unsigned int GetSamplingFrequencyIndex(unsigned int sampling_frequency) { ... }
static void
MakeAdtsHeader(unsigned char *bits,
               size_t  frame_size,
               unsigned int  sampling_frequency_index,
               unsigned int  channel_configuration) { ... }

static const size_t kAdtsHeaderLength = 7;

- (void)appendAudioSampleBuffer2:(CMSampleBufferRef)sampleBuffer
{
    // Get the actual audio data from the block buffer.
    CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
    size_t blockBufferLength = CMBlockBufferGetDataLength(blockBuffer);

    // Get the audio meta-data from its AudioFormatDescRef
    CMAudioFormatDescriptionRef audioFormat = CMSampleBufferGetFormatDescription(sampleBuffer);
    const AudioStreamBasicDescription *asbd = CMAudioFormatDescriptionGetStreamBasicDescription(audioFormat);

    // These are the values we will need to build our ADTS header
    unsigned int sample_rate = asbd->mSampleRate;
    unsigned int channel_count = asbd->mChannelsPerFrame;
    unsigned int sampling_frequency_index = GetSamplingFrequencyIndex(sample_rate);
    unsigned int channel_configuration = channel_count;

    // Create a byte buffer with first the header, and then the sample data.
    NSMutableData *buffer = [NSMutableData dataWithLength:kAdtsHeaderLength + blockBufferLength];
    MakeAdtsHeader((unsigned char*)[buffer mutableBytes], blockBufferLength, sampling_frequency_index, channel_configuration);
    CMBlockBufferCopyDataBytes(blockBuffer, 0, blockBufferLength, ((char*)[buffer mutableBytes])+kAdtsHeaderLength);

    // Calculate a timestamp int64 that Bento4 can use, by converting our CMTime into an Int64 in the timescale of the audio stream.
    CMTime presentationTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer);
    AP4_UI64 ts = CMTimeConvertScale(presentationTime, _audioStream->m_TimeScale, kCMTimeRoundingMethod_Default).value;

    _audioStream->WritePES(
        (const unsigned char*)[buffer bytes],
        (unsigned int)[buffer length],
        ts,
        false, // don't need a decode timestamp for audio
        ts,
        true, // do write a presentation timestamp so we can sync a/v
        *_output
    );
}

这篇关于多路复用附件B MPEG-TS时，是否等同于SPS和PPS的音频?什么是"DecoderInfo"?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！