使用Core Audio获取麦克风输入和扬声器输出 [英] Getting mic input and speaker output using Core Audio

问题描述

所以我最近调查了核心数据并且仍然是新手。我无法理解我正在使用哪些数据以及它如何影响整体数据流。所以对于某些背景，我有一个应用程序，使用webRTC在手机之间进行视频/音频流。但是，我想查看通过麦克风输入设备的数据和通过扬声器输出的数据。我查看了AurioTouch演示和Core Audio，目前我有这个：

So I looked into core data recently a bit and am still a newbie. I have trouble understanding what data I am taping into and how it is effecting the overall data flow. So for some background, I have an app that does video/audio streaming between phones using webRTC. However, I want to check out the data that is being inputed into the device through my mic and the data outputted through the speaker. I looked into AurioTouch demo and Core Audio and currently I have this:

- (void)setupIOUnit
{
    // Create a new instance of AURemoteIO

    AudioComponentDescription desc;
    desc.componentType = kAudioUnitType_Output;
    desc.componentSubType = kAudioUnitSubType_RemoteIO;
    desc.componentManufacturer = kAudioUnitManufacturer_Apple;
    desc.componentFlags = 0;
    desc.componentFlagsMask = 0;

    AudioComponent comp = AudioComponentFindNext(NULL, &desc);
    AudioComponentInstanceNew(comp, &rioUnit);

    //  Enable input and output on AURemoteIO
    //  Input is enabled on the input scope of the input element
    //  Output is enabled on the output scope of the output element

    UInt32 one = 1;
    AudioUnitSetProperty(rioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Input, 1, &one, sizeof(one));
    AudioUnitSetProperty(rioUnit, kAudioOutputUnitProperty_EnableIO, kAudioUnitScope_Output, 0, &one, sizeof(one));


    // Set the MaximumFramesPerSlice property. This property is used to describe to an audio unit the maximum number
    // of samples it will be asked to produce on any single given call to AudioUnitRender
    UInt32 maxFramesPerSlice = 4096;
    AudioUnitSetProperty(rioUnit, kAudioUnitProperty_MaximumFramesPerSlice, kAudioUnitScope_Global, 0, &maxFramesPerSlice, sizeof(UInt32));

    // Get the property value back from AURemoteIO. We are going to use this value to allocate buffers accordingly
    UInt32 propSize = sizeof(UInt32);
    AudioUnitGetProperty(rioUnit, kAudioUnitProperty_MaximumFramesPerSlice, kAudioUnitScope_Global, 0, &maxFramesPerSlice, &propSize);

    // Set the render callback on AURemoteIO
    AURenderCallbackStruct renderCallback;
    renderCallback.inputProc = performRender;
    renderCallback.inputProcRefCon = NULL;
    AudioUnitSetProperty(rioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &renderCallback, sizeof(renderCallback));

    NSLog(@"render set now");
    // Initialize the AURemoteIO instance
    AudioUnitInitialize(rioUnit);
    [self startIOUnit];
    return;
}

- (OSStatus)startIOUnit
{
    OSStatus err = AudioOutputUnitStart(rioUnit);
    if (err) NSLog(@"couldn't start AURemoteIO: %d", (int)err);
    return err;
}

渲染回调函数

static OSStatus performRender (void                         *inRefCon,
                           AudioUnitRenderActionFlags   *ioActionFlags,
                           const AudioTimeStamp         *inTimeStamp,
                           UInt32                       inBusNumber,
                           UInt32                       inNumberFrames,
                           AudioBufferList              *ioData)
{
    OSStatus err = noErr;
//    the data gets rendered here

    err = AudioUnitRender(rioUnit, ioActionFlags, inTimeStamp, 1, inNumberFrames, ioData);

    if (ioData->mBuffers[0].mDataByteSize >= 12) {
        NSData *myAudioData = [NSData dataWithBytes: ioData->mBuffers[0].mData length:12];
        NSLog(@" playback's first 12 bytes: %@", myAudioData);
    }

    for (UInt32 i=0; i<ioData->mNumberBuffers; ++i) {
        memset(ioData->mBuffers[i].mData, 0, ioData->mBuffers[i].mDataByteSize);
    }

    return err;
}

这打印出一些数据，此时我不知道是否是是麦克风输入或扬声器输出。让我感到不安的是，即使清除了ioData的缓冲区，我仍然可以在另一部手机上获取音频并播放另一部手机发送的音频。这有点告诉我，我既不接触麦克风输入也不接触扬声器输出。

This prints out some data, which I do not know at this point whether it is the microphone input, or the speaker output. What disturbs me is that even after clearing ioData's buffer, I am still getting audio on the other phone and can play the audio sent by the other phone. This kinda suggests to me that I am touching neither the mic input nor the speaker output.

我看到这一行有一些不同的参数：

I have seen some varying parameters for this line:

AudioUnitSetProperty(rioUnit, kAudioUnitProperty_SetRenderCallback, kAudioUnitScope_Input, 0, &renderCallback, sizeof(renderCallback));

我想知道我是否有这些错误或其他什么。另外，是这一行：

and I am wondering if I just have these wrong or something. In addition, is this line:

err = AudioUnitRender(rioUnit, ioActionFlags, inTimeStamp, 1, inNumberFrames, ioData);

受AudioUnitSetProperty的影响？在这种情况下设置1做了什么？

influenced by AudioUnitSetProperty? What does setting the 1 do in this scenario?

任何帮助都会很精彩。理想情况下，我希望能够将扬声器输出数据（可能是文件）以及麦克风输入进行采样。

Any help would he wonderful. Ideally, I want to be able to sample the speaker output data (maybe into a file) as well as the microphone input.

推荐答案

远程IO音频单元是核心音频的一部分，可同时进行输入和输出。它是一个完整的单元，可以录制/播放硬件（麦克风/扬声器）和/或程序中的音频。
这有时让人感到困惑。这样想吧。

The Remote IO audio unit is a part in core audio that does both input and output. It is a complete unit that can record/play audio either the hardware (mic/speaker) and/or from your program. That sometimes makes it confusing. Think of it this way.

**您在远程IO设备上有输入和输出。

**您还可以在远程IO设备上使用软件和硬件。
硬件输入是Mic。
硬件输出是扬声器。
软件输入是以编程方式创建的波形。
软件输出是已创建的波形。

** You have inputs and outputs on a remote IO unit.
** You also software and hardware on a remote IO unit. The hardware input is the Mic. The hardware output is the Speaker. The software input is a waveform you create programmatically. The software output is a waveform that has been created.

-------输入------------

-------Inputs------------

总线0：从应用程序读取（以编程方式构建音频波形）。在这里编写一个定期自动调用的回调。它说给我下一个音频样本。例如，你的代码可以为你提供以编程方式生成的三角波的音频样本。

因此你生成一个波形进入程序。您也可以从其他音频设备的输出中提供此输入。

Bus 0: read from your application (You construct audio waveforms programmatically). Here you write a callback that is automatically called periodically. It says "give me the next audio samples." For example, your code in there could give it audio samples of a triangle wave that you generate programmatically.
So you generate a waveform to feed into the program. You can also feed this input from the output of some other audio unit.

总线1：从麦克风读取。在这里，您可以从麦克风中读取音频样本。请注意，这些只是原始样本。您可以选择将它们保存到文件（例如录制应用程序），通过网络发送，甚至将它们连接到扬声器（见下文）。你不会听到来自麦克风的声音....它不会保存......除非你用它做点什么。

Bus 1: read from the microphone. Here you can read audio samples from the microphone. Note that these are just the raw samples. You can choose to either save them to a file (eg recording app), send them over a network, or even connect them to the speaker (see below). You wont hear the audio from the mic.... it wont save.... unless YOU do something with it.

--------- - 输出----------

----------Outputs----------

总线0：电话扬声器。在这里，您可以编写音频数据，它将在扬声器上播放。所以你得到另一个回调，上面写着给我播放样本，然后用音频填充缓冲区并播放它。在当前缓冲区完成播放之前的某个时间周期性地发生回调。

Bus 0: phone speaker. Here you can write audio data and it will play them on the speaker. So you get another callback that says "give me samples to play" and you fill up the buffer with audio and it plays it. The callback happens periodically sometime before the current buffer is finished playing.

总线1：写入您的应用。在这里，您可以获取远程IO生成的音频，并在您的应用中使用它做一些事情。例如，您可以将输出连接到另一个或将数据写入文件。

Bus 1: write to your app. Here you can take the audio generated by the remote IO and do something with it in your app. For example, you can connect the output to another or write the data to a file.

所以回答您的问题在这种情况下设置1做了什么？

So to answer your question "What does setting the 1 do in this scenario?"

以下是来自Apple在AudioUnitRender上的规范

Here is the spec from apple on AudioUnitRender

OSStatus AudioUnitRender ( AudioUnit inUnit, 
AudioUnitRenderActionFlags *ioActionFlags, 
const AudioTimeStamp *inTimeStamp,
UInt32 inOutputBusNumber,
UInt32 inNumberFrames,
AudioBufferList *ioData );

1表示您正在读取输出上的总线1，即生成的音频帧。所以你可以在这里做任何你想做的事情，它不会影响演讲者播放的内容。

The 1 says that you are reading bus 1 on the output which is audio frames produced. So you can do whatever you want here and it will NOT affect what is played at the speaker.

如果要检查麦克风数据，请在输入上使用总线1。如果你想检查扬声器数据，请在输出上使用总线0。

If you want to examine the mic data, use bus 1 on the input. If you want to examine speaker data, use bus 0 on the output.

请注意，你不能在回调中花费很长时间。不建议做任何可能花费大量时间的事情（例如写入网络，写入文件，打印）。在这种情况下，您可以使用GCD或类似的东西。

Note that you cannot do things that take a long time in the callbacks. It is not advised to do anything that could take a lot of time (such as write to network, write to file, print). In that case, you could use GCD or something similar.

这篇关于使用Core Audio获取麦克风输入和扬声器输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！