如何使用 CoreAudio 的 AudioConverter 实时编码 AAC? [英] How do I use CoreAudio's AudioConverter to encode AAC in real-time?

问题描述

我能找到的所有使用 AudioConverterRef 的示例代码都专注于我预先拥有所有数据的用例(例如转换磁盘上的文件).他们通常使用要转换的 PCM 调用 AudioConverterFillComplexBuffer 作为 inInputDataProcUserData 并在回调中填写它.(这真的是它应该如何使用吗?为什么它需要回调?)对于我的用例，我试图从麦克风流式传输 aac 音频，所以我没有文件，我的 PCM 缓冲区正在实时填写.

All the sample code I can find that uses AudioConverterRef focuses on use cases where I have all the data up-front (such as converting a file on disk). They commonly call AudioConverterFillComplexBuffer with the PCM to be converted as the inInputDataProcUserData and just fill it in in the callback. (Is that really how it's supposed to be used? Why does it need a callback, then?) For my use case, I'm trying to stream aac audio from the microphone, so I have no file, and my PCM buffer is being filled in in real time.

由于我没有预先获得所有数据，因此我尝试在输入数据输出后在回调中执行 *ioNumberDataPackets = 0 ，但这只是将 AudioConverter 置于死状态需要AudioConverterReset()ted，我没有从中得到任何数据.

Since I don't have all the data up-front, I've tried doing *ioNumberDataPackets = 0 in the callback once my input data is out, but that just puts the AudioConverter in a dead state where it needs to be AudioConverterReset()ted, and I don't get any data out of it.

我在网上看到建议的一种方法是，如果我存储的数据太小，则从回调中返回错误，并在有更多数据时再试一次，但这似乎浪费了资源，以至于我甚至无法让自己尝试.

One approach I've seen suggested online is to return an error from the callback if the data I have stored is too small, and just try again once I have more data, but that seems like such a waste of resources that I can't bring myself to even try it.

我真的需要重试直到我的输入缓冲区足够大"，还是有更好的方法?

Do I really need to do the "retry until my input buffer is big enough", or is there a better way?

推荐答案

AudioConverterFillComplexBuffer 实际上并不意味着用我在这里的输入缓冲区填充编码器".这意味着用来自编码器的编码数据填充这个输出缓冲区".从这个角度来看，回调突然变得有意义——它用于获取源数据以满足为我填充这个输出缓冲区"的请求.也许这对其他人来说是显而易见的，但我花了 很长时间 来理解这一点(从所有 AudioConverter 示例代码中，我看到人们通过 inInputDataProcUserData 发送输入数据的地方浮动，我猜我不是唯一一个).

AudioConverterFillComplexBuffer does not actually mean "fill the encoder with my input buffer that I have here". It means "fill this output buffer here with encoded data from the encoder". With this perspective, the callback suddenly makes sense -- it is used to fetch source data to satisfy the "fill this output buffer for me" request. Maybe this is obvious to others, but it took me a long time to understand this (and from all the AudioConverter sample code I see floating around where people send input data through inInputDataProcUserData, I'm guessing I'm not the only one).

AudioConverterFillComplexBuffer 调用是阻塞的，并期望您从回调同步向其传送数据.如果您是实时编码，则需要在您自己设置的单独线程上调用 FillComplexBuffer.在回调中，您可以检查可用的输入数据，如果不可用，则需要在信号量上进行阻塞.使用 NSCondition，编码器线程看起来像这样:

The AudioConverterFillComplexBuffer call is blocking, and is expecting you to deliver data to it synchronously from the callback. If you are encoding in real time, you will thus need to call FillComplexBuffer on a separate thread that you set up yourself. In the callback, you can then check for available input data, and if it is not available, you need to block on a semaphore. Using an NSCondition, the encoder thread would then look something like this:

- (void)startEncoder
{
    OSStatus creationStatus = AudioConverterNew(&_fromFormat, &_toFormat, &_converter);

    _running = YES;
    _condition = [[NSCondition alloc] init];
    [self performSelectorInBackground:@selector(_encoderThread) withObject:nil];
}

- (void)_encoderThread
{
    while(_running) {
        // Make quarter-second buffers.
        size_t bufferSize = (_outputBitrate/8) * 0.25;
        NSMutableData *outAudioBuffer = [NSMutableData dataWithLength:bufferSize];
        AudioBufferList outAudioBufferList;
        outAudioBufferList.mNumberBuffers = 1;
        outAudioBufferList.mBuffers[0].mNumberChannels = _toFormat.mChannelsPerFrame;
        outAudioBufferList.mBuffers[0].mDataByteSize = (UInt32)bufferSize;
        outAudioBufferList.mBuffers[0].mData = [outAudioBuffer mutableBytes];

        UInt32 ioOutputDataPacketSize = 1;

        _currentPresentationTime = kCMTimeInvalid; // you need to fill this in during FillComplexBuffer
        const OSStatus conversionResult = AudioConverterFillComplexBuffer(_converter, FillBufferTrampoline, (__bridge void*)self, &ioOutputDataPacketSize, &outAudioBufferList, NULL);

        // here I convert the AudioBufferList into a CMSampleBuffer, which I've omitted for brevity.
        // Ping me if you need it.
        [self.delegate encoder:self encodedSampleBuffer:outSampleBuffer];
    }
}

回调可能如下所示:(注意，我通常使用此蹦床立即转发到我的实例上的方法(通过在 inUserData 中转发我的实例；为简洁起见，省略了此步骤)):

And the callback could look like this: (note that I normally use this trampoline to immediately forward to a method on my instance (by forwarding my instance in inUserData; this step is omitted for brevity)):

static OSStatus FillBufferTrampoline(AudioConverterRef               inAudioConverter,
                                        UInt32*                         ioNumberDataPackets,
                                        AudioBufferList*                ioData,
                                        AudioStreamPacketDescription**  outDataPacketDescription,
                                        void*                           inUserData)
{
    [_condition lock];

    UInt32 countOfPacketsWritten = 0;

    while (true) {
        // If the condition fires and we have shut down the encoder, just pretend like we have written 0 bytes and are done.
        if(!_running) break;

        // Out of input data? Wait on the condition.
        if(_inputBuffer.length == 0) {
            [_condition wait];
            continue;
        }

        // We have data! Fill ioData from your _inputBuffer here.
        // Also save the input buffer's start presentationTime here.

        // Exit out of the loop, since we're done waiting for data
        break;
    }

    [_condition unlock];

        // 2. Set ioNumberDataPackets to the amount of data remaining


    // if running is false, this will be 0, indicating EndOfStream
    *ioNumberDataPackets = countOfPacketsWritten;

    return noErr;
}

为了完整起见，以下是您如何向该编码器提供数据以及如何正确关闭它:

And for completeness, here's how you would then feed this encoder with data, and how to shut it down properly:

- (void)appendSampleBuffer:(CMSampleBufferRef)sampleBuffer
{
    [_condition lock];
    // Convert sampleBuffer and put it into _inputBuffer here
    [_condition broadcast];
    [_condition unlock];
}

- (void)stopEncoding
{
    [_condition lock];
    _running = NO;
    [_condition broadcast];
    [_condition unlock];
}

这篇关于如何使用 CoreAudio 的 AudioConverter 实时编码 AAC?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！