播放来自CMSampleBuffer的音频 [英] Play audio from CMSampleBuffer

问题描述

我为iOS中的群组创建了视频聊天应用.我一直在寻找一些方法来分别控制不同参与者的音量.我在 RemoteAudioTrack 中找到了使用 isPlaybackEnabled 进行静音和取消静音的方法，但没有控制音量.

I have created a video chat app for groups in iOS. I have been searching for some ways to control the audio volume for different participant separately. I found way to mute and unmute using isPlaybackEnabled in RemoteAudioTrack, but not to control volume.

我还认为我们是否可以在 AVAudioPlayer 中使用它.我找到了 addSink .这是我在此处尝试过的操作:

I also thought if we can use it in AVAudioPlayer. I found addSink. This is what I tried from here:

class Audio: NSObject, AudioSink {
    var a = 1
    func renderSample(_ audioSample: CMSampleBuffer!) {
        print("audio found", a)
        a += 1

        var audioBufferList = AudioBufferList()
        var data = Data()
        var blockBuffer : CMBlockBuffer?

        CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(audioSample, bufferListSizeNeededOut: nil, bufferListOut: &audioBufferList, bufferListSize: MemoryLayout<AudioBufferList>.size, blockBufferAllocator: nil, blockBufferMemoryAllocator: nil, flags: 0, blockBufferOut: &blockBuffer)
        let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))

        for audioBuffer in buffers {
            let frame = audioBuffer.mData?.assumingMemoryBound(to: UInt8.self)
            data.append(frame!, count: Int(audioBuffer.mDataByteSize))
        }

        let player = try! AVAudioPlayer(data: data) //crash here
        player.play()
    }
}

但是它在 let player上崩溃了=试试！AVAudioPlayer(数据:数据).

这是错误:致命错误:尝试！"表达式意外引发错误:Error Domain = NSOSStatusErrorDomain Code = -39(null)":文件.

这是 data ，所以我想它没有被转换:

This is data so I guess it is not converted:

▿ 0 bytes
  - count : 0
  ▿ pointer : 0x000000016d7ae160
    - pointerValue : 6131736928
  - bytes : 0 elements

这是 audioSample :

<CMAudioFormatDescription 0x2815a3de0 [0x1bb2ef830]> {
    mediaType:'soun' 
    mediaSubType:'lpcm' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 16000.000000 
            mFormatID: 'lpcm' 
            mFormatFlags: 0xc 
            mBytesPerPacket: 2 
            mFramesPerPacket: 1 
            mBytesPerFrame: 2 
            mChannelsPerFrame: 1 
            mBitsPerChannel: 16     } 
        cookie: {(null)} 
        ACL: {(null)}
        FormatList Array: {(null)} 
    } 
    extensions: {(null)}
}

推荐答案

您可以从 CMSampleBuffer 获取完整的数据缓冲区，并将其转换为 Data :

You can get the full data buffer from CMSampleBuffer and convert it to Data:

let blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer)
let blockBufferDataLength = CMBlockBufferGetDataLength(blockBuffer!)
var blockBufferData  = [UInt8](repeating: 0, count: blockBufferDataLength)
let status = CMBlockBufferCopyDataBytes(blockBuffer!, atOffset: 0, dataLength: blockBufferDataLength, destination: &blockBufferData)
guard status == noErr else { return }
let data = Data(bytes: blockBufferData, count: blockBufferDataLength)

另请参阅 AVAudioPlayer 概述:

除非要播放从网络流捕获的音频或需要非常低的I/O延迟，否则请使用该类进行音频播放.

Use this class for audio playback unless you are playing audio captured from a network stream or require very low I/O latency.

所以我认为这不会为您服务.您最好使用 AVAudioEngine 或

So I don't think it will work for you. You should better use AVAudioEngine or Audio Queue Services.

这篇关于播放来自CMSampleBuffer的音频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！