Android 上的低延迟音频播放 [英] Low-latency audio playback on Android
问题描述
我目前正在尝试最小化简单应用程序的音频延迟:
I'm currently attempting to minimize audio latency for a simple application:
我在 PC 上有一个视频，我正在通过 RTP 将视频的音频传输到移动客户端.使用非常相似的缓冲算法，我可以在 iOS 上实现 90 毫秒的延迟，但在 Android 上却达到了可怕的 ±180 毫秒.
I have a video on a PC, and I'm transmitting the video's audio through RTP to a mobile client. With a very similar buffering algorithm, I can achieve 90ms of latency on iOS, but a dreadful ±180ms on Android.
我猜测差异源于众所周知的延迟
然而，在阅读了一些之后，我看到了这篇文章，其中指出:
However, after reading around for a bit, I came upon this article, which states that:
自 Android 4.1/4.2 起，某些设备可使用低延迟音频.
Low-latency audio is available since Android 4.1/4.2 in certain devices.
低延迟音频可以使用 libpd，这是 Android 的纯数据库.
Low-latency audio can be achieved using libpd, which is Pure Data library for Android.
我有 2 个问题，与这 2 个陈述直接相关:
I have 2 questions, directly related to those 2 statements:
在哪里可以找到有关 Jellybean 中新的低延迟音频的更多信息?我能找到的就这些了，但非常缺乏具体信息.这些更改对我来说应该是透明的，还是应该实现一些新的类/API 调用来让我注意到我的应用程序中的任何更改?我正在使用 AudioTrack API，我什至不确定它是否应该从这种改进中受益，或者我是否应该研究其他一些音频播放机制.
Where can I find more information on the new low-latency audio in Jellybean? This is all I can find but it's sorely lacking in specific information. Should the changes be transparent to me, or is there some new class/API calls I should be implementing for me to notice any changes in my application? I'm using the AudioTrack API, and I'm not even sure if it should reap benefits from this improvement or if I should be looking into some other mechanism for audio playback.
我应该考虑使用 libpd 吗?在我看来，这是我实现较低延迟的唯一机会，但由于我一直认为 PD 是一种音频合成实用程序，它是否真的适合仅从网络流中抓取帧并回放的项目?我真的没有做任何合成.我是不是走错路了?
Should I look into using libpd? It seems to me like it's the only chance I have of achieving lower latencies, but since I've always thought of PD as an audio synthesis utility, is it really suited for a project that just grabs frames from a network stream and plays them back? I'm not really doing any synthesizing. Am I following the wrong trail?
作为补充说明，在有人提到 OpenSL ES 之前，这篇文章很清楚地表明使用它不会改善延迟:
As an additional note, before someone mentions OpenSL ES, this article makes it quite clear that no improvements in latency should be expected from using it:
由于 OpenSL ES 是本机 C API，非 Dalvik 应用程序线程调用 OpenSL ES 没有 Dalvik 相关的开销，例如垃圾收集暂停.但是，没有额外的性能优势使用除此之外的 OpenSL ES.特别是使用 OpenSLES 不会导致更低的音频延迟，更高的调度优先级，等等，而不是平台通常提供的内容."
推荐答案
为了在 Android 4.2.2 版本中实现最低延迟，您应该按照从最不明显到最明显的顺序执行以下操作:
For lowest latency on Android as of version 4.2.2, you should do the following, ordered from least to most obvious:
如果可能，选择支持 FEATURE_AUDIO_PRO 的设备，否则选择 FEATURE_AUDIO_LOW_LATENCY.(低延迟"是一种方式 50 毫秒；专业是 <20 毫秒往返.)
Pick a device that supports FEATURE_AUDIO_PRO if possible, or FEATURE_AUDIO_LOW_LATENCY if not. ("Low latency" is 50ms one way; pro is <20ms round trip.)
使用 OpenSL.Dalvik GC 的摊销成本较低，但运行时需要的时间比低延迟音频线程允许的时间长.
Use OpenSL. The Dalvik GC has a low amortized cost, but when it runs it takes more time than a low-latency audio thread can allow.
在缓冲队列回调中处理音频.系统在比普通用户模式线程具有更有利调度的线程中运行缓冲区队列回调.
Process audio in a buffer queue callback. The system runs buffer queue callbacks in a thread that has more favorable scheduling than normal user-mode threads.
使您的缓冲区大小成为 AudioManager.getProperty(PROPERTY_OUTPUT_FRAMES_PER_BUFFER) 的倍数.否则，您的回调偶尔会在每个时间片中收到两次调用而不是一次调用.除非您的 CPU 使用率非常低，否则这可能最终会出现故障.(在 Android M 上，由于缓冲区处理代码中的错误，准确使用系统缓冲区大小非常重要.)
Make your buffer size a multiple of AudioManager.getProperty(PROPERTY_OUTPUT_FRAMES_PER_BUFFER). Otherwise your callback will occasionally get two calls per timeslice rather than one. Unless your CPU usage is really light, this will probably end up glitching. (On Android M, it is very important to use EXACTLY the system buffer size, due to a bug in the buffer handling code.)
使用 AudioManager.getProperty(PROPERTY_OUTPUT_SAMPLE_RATE) 提供的采样率.否则，您的缓冲区会绕过系统重采样器.
Use the sample rate provided by AudioManager.getProperty(PROPERTY_OUTPUT_SAMPLE_RATE). Otherwise your buffers take a detour through the system resampler.
永远不要在缓冲区回调中进行系统调用或锁定同步对象.如果必须同步，请使用无锁结构.为获得最佳结果，请使用完全无需等待的结构，例如单读取器单写入器环形缓冲区.许多开发人员都犯了这个错误，最终出现了不可预测且难以调试的故障.
Never make a syscall or lock a synchronization object inside the buffer callback. If you must synchronize, use a lock-free structure. For best results, use a completely wait-free structure such as a single-reader single-writer ring buffer. Loads of developers get this wrong and end up with glitches that are unpredictable and hard to debug.
使用向量指令，例如 NEON、SSE 或目标处理器上的任何等效指令集.
Use vector instructions such as NEON, SSE, or whatever the equivalent instruction set is on your target processor.
测试和衡量您的代码.跟踪运行所需的时间——并记住您需要知道最坏情况下的性能，而不是平均值，因为最坏情况是导致故障的原因.并且要保守.您已经知道，如果处理音频所需的时间比播放时间长，您将永远无法获得低延迟.但在 Android 上，这一点更为重要，因为 CPU 频率波动很大.您可以将 60-70% 的 CPU 用于音频，但请记住，随着设备变得越来越热或越来越冷，或者随着 wifi 或 LTE 无线电的启动和停止等，这种情况会发生变化.
Test and measure your code. Track how long it takes to run--and remember that you need to know the worst-case performance, not the average, because the worst case is what causes the glitches. And be conservative. You already know that if it takes more time to process your audio than it does to play it, you'll never get low latency. But on Android this is even more important, because the CPU frequency fluctuates so much. You can use perhaps 60-70% of CPU for audio, but keep in mind that this will change as the device gets hotter or cooler, or as the wifi or LTE radios start and stop, and so on.
低延迟音频不再是 Android 的新功能，但它仍然需要在硬件、驱动程序、内核和框架中进行特定于设备的更改才能实现.这意味着您可以预期不同设备的延迟会有很大差异，并且考虑到 Android 手机销售的不同价位，可能总会存在差异.查找 FEATURE_AUDIO_PRO 或 FEATURE_AUDIO_LOW_LATENCY 以确定满足您的应用要求的延迟标准的设备.
Low-latency audio is no longer a new feature for Android, but it still requires device-specific changes in the hardware, drivers, kernel, and framework to pull off. This means that there's a lot of variation in the latency you can expect from different devices, and given how many different price points Android phones sell at, there probably will always be differences. Look for FEATURE_AUDIO_PRO or FEATURE_AUDIO_LOW_LATENCY to identify devices that meet the latency criteria your app requires.
这篇关于Android 上的低延迟音频播放的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
