带有AVAudioConverterInputBlock的AVAudioConverter在处理后口吃音频 [英] AVAudioConverter with AVAudioConverterInputBlock stutters audio after processing

问题描述

我正在尝试将音频缓冲区转换为其他格式，并且正在使用AVAudioConverter.当您具有相同的采样率并且不需要使用AVAudioConverterInputBlock时，AVAudioConverter会执行此工作.

I'm trying to convert audio buffers to a different format, and I'm using AVAudioConverter. AVAudioConverter does the job when you have the same sample rate and you don't need to use the AVAudioConverterInputBlock.

但是，如果我使用相同的采样率，音频数据就会出现奇怪的卡顿现象.我感觉我没有很好地处理输入块.输出的单词重复两次或三次.下面是完整的方法:

But if I'm dealing with the same sample rate, I'm getting a strange stutter in my audio data. I have a feeling I'm not handling the input block well. The output has words repeating two or three times. Below is the full method:

func sendAudio(audioFile: URL, completionHandler: @escaping (Bool, Bool, Data?)->Void) {

    createSession(){ sessionUrl, observeURL, session in
        let file = try! AVAudioFile(forReading: audioFile)
        let formatOfAudio = file.processingFormat
        self.engine = AVAudioEngine()
        guard let input = self.engine.inputNode else {
            print("no input")
            return
        }
        //The audio in format in this case is: <AVAudioFormat 0x61800009d010:  2 ch,  44100 Hz, Float32, non-inter>
        let formatIn = formatOfAudio
        let formatOut = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: 16000, channels: 1, interleaved: true)
        let mixer = AVAudioMixerNode()
        self.engine.attach(mixer)
        mixer.volume = 0.0
        self.engine.attach(self.audioPlayerNode)
        self.engine.connect(self.audioPlayerNode, to: mixer, format: formatIn)
        self.engine.connect(input, to: mixer, format: input.outputFormat(forBus: 0))
        self.engine.connect(mixer, to: self.engine.mainMixerNode, format: formatIn)
        let audioConverter = AVAudioConverter(from: formatIn, to: formatOut)
        mixer.installTap(onBus: 0, bufferSize: 32000, format: formatIn, block: {
            (buffer: AVAudioPCMBuffer!, time: AVAudioTime!) -> Void in
                let convertedBuffer = AVAudioPCMBuffer(pcmFormat: formatOut, frameCapacity: buffer.frameCapacity)
                let inputBlock: AVAudioConverterInputBlock = { inNumPackets, outStatus in
                    outStatus.pointee = AVAudioConverterInputStatus.haveData
                    return buffer
                }
                var error: NSError? = nil
                let status = audioConverter.convert(to: convertedBuffer, error: &error, withInputFrom: inputBlock)
                let myData = convertedBuffer.toData()
                completionHandler(true, false, myData)
        })
        self.audioPlayerNode.scheduleFile(file, at: nil){
            self.delayWithSeconds(3.0){
            self.engine.stop()
            mixer.removeTap(onBus: 0)
            completionHandler(true, true, nil)
            }
        }
        do {
            try self.engine.start()
        } catch {
            print(error)
        }
        self.audioPlayerNode.play()
    }
}

有什么想法吗?我从 Apple幻灯片示例中获得了这段代码:

Any thoughts? I got this code from an Apple slide sample:

// Create an input block that’s called when converter needs input
let inputBlock : AVAudioConverterInputBlock = {inNumPackets, outStatus in 
    if (<no_data_available>) {   
        outStatus.memory = AVAudioConverterInputStatus.NoDataNow; 
        return nil;  
    } else if (<end_of_stream>) {   
        outStatus.memory = AVAudioConverterInputStatus.EndOfStream; 
        return nil;  
    } else {
        ..outStatus.memory = AVAudioConverterInputStatus.HaveData;   
        return inBuffer; // fill and return input buffer 
    }  
}

推荐答案

对于发现此问题的任何人，真正的根本原因是对 AVAudioConverterInputBlock 的错误使用.只要足够大，目标缓冲区的容量就无关紧要，但是将重复调用该块，直到目标缓冲区被填满为止.

For anybody finding this, the actual underlying cause is the incorrect use of AVAudioConverterInputBlock. The destination buffer capacity doesn't matter as long as it is large enough, however the block will be called repeatedly until the destination buffer is filled.

如果源缓冲区包含 ABC ，它将用 ABCABCABC ... 填充目标.然后，如果将其通过管道传输到实时回放，则会随机切断这些块以适应回放时间，从而产生怪异的裂纹.

If your source buffer contains ABC, it will fill up the destination with ABCABCABC.... Then, if you pipe it to realtime playback, the chunks are getting cut off randomly to fit the playback timing, resulting in this weird crackle.

实际的解决方案是将缓冲区提交到转换器后，将 AVAudioConverterInputStatus 正确设置为 .noDataNow .请注意，返回 .endOfStream 将永远锁定转换器对象.

The actual solution is to properly set AVAudioConverterInputStatus to .noDataNow once the buffer is submitted to the converter. Note that returning .endOfStream will lock up the converter object forever.

var gotData = false
self.converter.convert(to: convertedBuffer, error: nil, withInputFrom: { (_, outStatus) -> AVAudioBuffer? in
    if gotData {
        outStatus.pointee = .noDataNow
        return nil
    }
    gotData = true
    outStatus.pointee = .haveData
    return inputBuffer
})            

这篇关于带有AVAudioConverterInputBlock的AVAudioConverter在处理后口吃音频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！