将低延迟音频从一台CoreAudio设备路由到另一台 [英] Routing low-latency audio from one CoreAudio device to another

问题描述

首先，提供一些背景信息:我正在编写一个MacOS/X应用程序，该应用程序使用CoreAudio从CoreAudio设备的输入流中接收音频信号，对音频进行一些实时处理，然后将其发送回该音频CoreAudio设备的输出流供用户收听.

First, some background info: I'm writing a MacOS/X application that uses CoreAudio to receive an audio signal from a CoreAudio device's input stream, do some real-time processing on the audio, and then send it back to that CoreAudio device's output stream for the user to hear.

此应用程序使用较低级别的CoreAudio API(即 AudioDeviceAddIOProc ， AudioDeviceStart 等-不是AudioUnits)来获取对用户指定的CoreAudio设备的独占访问权，将其设置为所需的采样率(96kHz)，然后执行其操作.它工作得很好，我对它的性能感到非常满意.

This application uses the lower-level CoreAudio APIs (i.e. AudioDeviceAddIOProc, AudioDeviceStart, etc -- not AudioUnits) to grab exclusive access to a user-specified CoreAudio device, set it to the desired sample rate (96kHz), and do its thing. It works very well, and I'm quite happy with its performance.

但是，我的程序当前有一个限制-一次只能使用一个CoreAudio设备.我想做的是扩展我的应用程序，以便用户可以彼此独立地选择其输入CoreAudio设备"和输出CoreAudio设备"，而不是仅限于使用提供两种功能的单个CoreAudio设备.输入音频源和输出音频接收器.

However, my program currently has a limitation -- it can only use a single CoreAudio device at a time. What I'd like to do is extend my application so that the user can choose his "input CoreAudio device" and his "output CoreAudio device" independently of each other, rather than being restricted to using only a single CoreAudio device that supplies both the input audio source and the output audio sink.

我的问题是，建议采用什么技术?我可以要求两个CoreAudio设备都可以设置为相同的采样率，但是即使我这样做了，我仍然必须处理各种问题，例如:

My question is, what is the recommended technique for doing this? I can require that both CoreAudio devices be settable to the same sample-rate, but even once I do that, I think I will have to handle various issues, such as:

• 从两个设备集成单独的 AudioDeviceStart()发起的回调，我怀疑不会以任何明确定义的顺序调用，甚至可能会相对于每个设备同时调用其他(?).我需要以某种方式将音频从一个回调传递到另一个回调，而理想情况下又不会显着增加音频延迟.

• integrating separate AudioDeviceStart()-initiated callbacks from the two devices, which I suspect will not be called in any well-defined order, and might even be called concurrently with respect to each other(?). I would need to pass audio from one callback to the other somehow, ideally without significantly increasing audio latency.

处理两个设备的采样时钟速率上的差异.例如.即使两个设备的标称采样率均设置为96kHz，我怀疑实际上可能是这种情况.上游设备以95.99999kHz的频率生成样本，而下游设备以96.000001kHz的频率生成样本(反之亦然)，这最终将导致我最终得到不够"或太多"的样本来馈送样本.给定的渲染回调过程中下游设备，会导致故障.

Handling differences in the sample-clock rates of the two devices. E.g. even if both devices are nominally set to 96kHz sample rate, I suspect it may actually be the case that e.g. the upstream device is producing samples at 95.99999kHz while the downstream device is consuming them at 96.000001kHz (or vice-versa), and that would eventually cause me to end up with either "not enough" or "too many" samples to feed the downstream device during a given rendering-callback, causing a glitch.

我还没有考虑过的其他陷阱

Any other gotchas they I haven't considered yet

其他MacOS/X程序如何处理这些问题?

How do other MacOS/X programs handle these issues?

推荐答案

前段时间，我用C玩了概念证明游乐场混音器.这还没有完成，但实际上是可行的.该库使用了最低的Core Audio API，因此确实使用了诸如 AudioDeviceCreateIOProcID 和 AudioObjectAddPropertyListener 之类的东西.

A time ago I played with a proof of concept playground audiomixer in C. Nothing of this is finished, but things do actually work. The library uses the lowest Core Audio API available, thus indeed with things like AudioDeviceCreateIOProcID and AudioObjectAddPropertyListener.

简而言之，这个游乐场使我可以使用MacOS已知的多个音频设备，并在它们之间路由一个或多个音频流，同时沿途通过不同类型的节点"(例如，考虑矩阵混合器节点).

In short, this playground allows me to use multiple audio devices known to MacOS and route one or more audio streams between them while passing through different kinds of "nodes" along the way (think of a matrix mixer node for example).

AudioDeviceStart()发起的回调将从不同的(随机)线程中触发每个回调.同样，将不会以确定的顺序调用回调.我还发现回调之间的差异可能相差很大(看似取决于提供/请求数据的音频设备).为了解决此问题，我使用了无锁(即使用原子计数器)环形缓冲区.

AudioDeviceStart() initiated callbacks will fire each from a different (random) thread. Also, the callbacks will not be called in a deterministic order. I also found that the difference between the callbacks can vary a lot (seemingly depending on the audio device providing/asking for data). To solve this problem I used a lock-free (that is, using atomic counters) ringbuffer.

您对不同时钟域的关注是非常真实的.以96KHz运行的两个设备将以不同的速度运行.这可能会持续很长时间，但最终其中之一将耗尽数据并开始出现故障.如果外部设备未在外部(例如使用word或ptp)同步在一起，则它们将在自己的时域中运行.要在不同时域之间传递音频，您必须对音频数据进行异步采样率转换.SRC将需要有可能以非常小的比率转换并一路调整.其中一项做得很好的就是 Soxr .在Core Audio的世界中，有一个VarispeedNode，它使您可以做基本相同的事情.async-src解决方案的最大缺点是它引入了延迟，但是您可以指定低延迟".

Your concern about different clock domains is very real. Two devices running at 96KHz will run at different speeds. This can go well for a long time, but eventually one of them is going to run out of data and start to glitch. If the external devices are not externally synchronised together, using for example word or ptp, they'll run in their own time domain. To pass audio between different time domains you'll have to async-sample-rate-convert the audio data. And the SRC will need to have the possibility to convert in very small ratios and adjust along the way. One of those doing this very well is Soxr. In the world of Core Audio there is a VarispeedNode, which allows you to do basically the same thing. Big disadvantage of the async-src solution is the latency it introduces, however maybe you could specify "low-latency".

在您的情况下，不同音频设备的同步将是最大的挑战.以我为例，我发现不同音频设备的回调差异太大，无法选择一个作为时钟主设备"，因此我通过仔细地计时处理周期来最终创建了一个独立的时域.为此，我使用了诸如 mach_wait_until()和 mach_absolute_time()之类的低级计时机制(没有太多文档).

In your case the synchronisation of the different audio devices will be the biggest challenge. In my case I found the callbacks of different audio devices vary too much to select one for being "clock-master" so I ended up creating a standalone time domain by carefully timing the execution of the processing cycle. For this I used low level timing mechanisms like mach_wait_until() and mach_absolute_time() (there's not much documentation on that).

但是，可能还有另一种解决方案.从CoreAudio框架查看 AudioHardware.h 中的文档，似乎有一种方法可以使用 AudioHardwareCreateAggregateDevice()以编程方式创建聚合设备.这使您可以让MacOS处理不同音频设备的同步.还要注意 kAudioAggregateDeviceIsPrivateKey 键，该键可让您创建聚合设备而无需将其发布到整个系统.因此，该设备不会出现在音频MIDI设置"中(我认为).另请注意，此键使聚合在创建它的进程停止运行时消失.可能不是您所需要的，但这将是使用多个音频设备实现的一种非常可靠的方法.如果我要再次编写软件，我肯定会研究这种同步方式.

However, there might be another solution. Looking at the documentation in AudioHardware.h from the CoreAudio framework, there seems to be a way to create an aggregate device programmatically using AudioHardwareCreateAggregateDevice(). This allows you to let MacOS handle the synchronisation of different audio devices. Also note the kAudioAggregateDeviceIsPrivateKey key which allows you to create an aggregate device without publishing it to the whole system. So, the device won't show up in Audio MIDI Setup (I think). Please also note that this key makes the aggregate disappearing when the process which created it stops running. It might or might not be what you need, but this would be a very robust way of implementing using multiple audio devices. If I were to write the software again, I definitely would look into this way of doing the synchronisation.

通常，在处理低延迟音频时，您想要实现尽可能确定性的行为.但我确定您知道这一点.

In general when dealing with low-latency audio you want to achieve the most deterministic behaviour possible. But I'm sure you are aware of this.

另一个难题是，Apple的开发人员网站上没有提供Core Audio api的文档(

Another gotcha is that the documentation of the Core Audio api is not available on Apple's developer website (https://developer.apple.com/documentation/coreaudio/core_audio_functions?language=objc). For that you'll have to dive into the headers of the Core Audio framework where you'll find a lot of useful documentation about using the API.

在我的机器上，标题位于:/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/CoreAudio.framework/Versions/A/Headers

On my machine the headers are located at: /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/CoreAudio.framework/Versions/A/Headers

http://atastypixel.com/blog/four-common-音频开发中的错误 http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-forthings https://developer.apple.com/library/archive/qa/qa1467/_index.html

这篇关于将低延迟音频从一台CoreAudio设备路由到另一台的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！