在 Android 中重定向音频/创建备用声音路径 [英] Redirecting audio / creating alternate sound paths in Android

问题描述

有没有人有在 Android 中重定向音频或创建新声音路径的经验(使用 OpenSL ES、ALSA 等)?最终目标是创建一个虚拟麦克风来取代外部麦克风，在那里人们可以播放音频文件，就像他们对着麦克风说话一样.使用 AudioSource.MIC 访问麦克风的应用程序应使用此备用流.它没有必要与语音通话一起工作，我相信实现这种功能更难，因为这一切都在无线电中完成.

Does anyone have experience (using OpenSL ES, ALSA, etc.) with redirecting audio or creating new sound paths in Android? The end goal is to create a virtual microphone to replace the external microphone, where one can play audio files as if they were speaking into the microphone. Applications accessing the microphone with AudioSource.MIC should use this alternate stream. It's not necessary for it to work with voice calls, I believe achieving that sort of functionality is harder as it's all done within the radio.

关于从哪里开始的任何想法?我已经对 OpenSL 和 ALSA 进行了一些研究，但看起来我需要打包新固件 (ROM) 才能定义自定义音频路径.如果可以避免，我想创建一个应用程序级解决方案.电话是扎根"的(有 su 二进制文件).为此，目标设备是三星 Galaxy S4 Google 版 (GT-i9505G).具体来说，我正在寻找 i9505G 的音频驱动程序配置/源代码或任何参考资料.

Any ideas on where to begin? I've done some research with OpenSL and ALSA, but it looks like I'll need to package new firmware (ROM) in order to define custom audio paths. If it can be avoided I'd like to create an application-level solution. The phones are 'rooted' (have su binaries). The target device for this is the Samsung Galaxy S4 Google Edition (GT-i9505G). Specifically I'm looking for audio driver configurations / source code or any references for the i9505G.

提前致谢！

编辑 - 我已经检查了 CyanogenMod 10.2 源代码树，以及 jfltexx 驱动程序和内核.以下是 kernel/samsung/jf/sound 的内容:http://pastebin.com/7vK8THcZ.这在任何地方都有记录吗?

edit - I've checked out the CyanogenMod 10.2 source tree, along with the jfltexx drivers and kernel. Here are the contents of kernel/samsung/jf/sound: http://pastebin.com/7vK8THcZ. Is this documented anywhere?

推荐答案

我曾经在基于 Qualcomm 的 APQ8064 平台(这似乎与目标设备中的平台几乎相同的平台)的手机上实现了您所追求的功能).以下是我从中可以回忆起的内容的摘要，因为我无法再访问我编写的代码，也无法访问可以轻松进行此类修改的环境.所以如果这个答案读起来像是一堆零碎的记忆，那是因为它就是这样.

I once implemented the functionality you're after on a phone based on Qualcomm's APQ8064 platform (which seems to be nearly the same platform as the one in your target device). Below is a summary of what I can recall from this, as I no longer have access to the code I wrote, or an environment where I can easily do these kinds of modifications. So if this answer reads like a mess of fragmentary memories, that's because that's exactly what it is.

此信息或多或少也适用于其他 Qualcomm 平台(如 MSM8960 或 MSM8974)，但很可能对其他供应商的平台(NVidia Tegra、Samsung Exynos、TI OMAP 等)完全无用.

This info may also apply more-or-less to other Qualcomm platforms (like the MSM8960 or MSM8974), but will most likely be completely useless for platforms from other vendors (NVidia Tegra, Samsung Exynos, TI OMAP, etc).

简短说明:我使用的方法意味着录音应用程序获取的音频将经过 Android 多媒体框架和/或平台的多媒体 DSP 中的混音/音量控制.因此，如果您以 75% 的音量播放某些内容，录制它，然后以 75% 的音量播放录音，最终听起来可能会非常安静.如果您想获得未处理的 PCM 数据(在解码之后，但在混音/音量控制之前)，您将不得不考虑其他一些方法，例如自定义 AudioFlinger，但这不是我尝试过的，也无法提供相关信息.

A brief note: The method I used means that the audio that the recording application gets will have gone through mixing / volume control in the Android multimedia framework and/or the platform's multimedia DSP. So if you're playing something at 75% volume, recording it, and then playing back the recording at 75% volume it might end up sounding pretty quiet. If you want to get unprocessed PCM data (after decoding, but before mixing / volume control) you'll have to look into some other approach, e.g. customizing the AudioFlinger, but that's not something I've tried or can provide info on.

一些感兴趣的地点:

平台的音频驱动程序.特别是 msm-pcm-routing.c 文件.

ALSA UCM(用例管理器)设置文件.这只是一个示例 UCM 设置文件.根据所使用的确切平台，这些文件有许多变体，因此您的名称可能略有不同(尽管它应该以 snd_soc_msm_ 开头)，其内容可能会也与我链接的那个略有不同.
Kitkat 及更高版本的注意事项: UCM 设置文件用于 Jellybean(可能还有 ICS).我的理解是这些设置已移至名为 mixer_paths.xml 在 Kitkat 上.内容几乎相同，只是格式不同.

The ALSA UCM (Use-Case Manager) settings file. This is just an example UCM settings file. There are many variants of these files depending on the exact platform used, so your's may have a slightly different name (though it should start with snd_soc_msm_), and its contents will probably also differ slightly from the one I linked to.
NOTE for Kitkat and later: The UCM settings files were used on Jellybean (and possibly ICS). My understanding is that these settings have been moved to a file named mixer_paths.xml on Kitkat. The contents are pretty much the same, just in a different format.

音频 HAL 代码.ALSA UCM 存在于 libalsa-intf 中，AudioHardware/AudioPolicyManager/ALSADevice 代码存在于 <代码>audio-alsa.请注意，此代码适用于 Jellybean，因为这是我熟悉的最新版本.Kitkat 上的目录结构(可能还有一些文件/类)有所不同.

The audio HAL code. The ALSA UCM is present in libalsa-intf, and the AudioHardware / AudioPolicyManager / ALSADevice code is present in audio-alsa. Note that this code is for Jellybean, since that's the lastest version that I'm familiar with. The directory structure (and possibly some of the files / classes) differs on Kitkat.

如果您打开 UCM 设置文件并搜索 "HiFiPROXY Rx"，您会发现如下内容:

If you open up the UCM settings file and search for "HiFiPROXY Rx" you'll find something like this:

SectionVerb
Name "HiFiPROXY Rx"

EnableSequence
    'AFE_PCM_RX Audio Mixer MultiMedia1':1:1
EndSequence

DisableSequence
    'AFE_PCM_RX Audio Mixer MultiMedia1':1:0
EndSequence

# ALSA PCMs
CapturePCM 0
PlaybackPCM 0
EndSection

这定义了一个动词(本质上是音频用例的基础；还有修饰符可以应用于动词之上，用于同步播放和录音)名称为 "HiFiPROXY Rx"(HiFi 绰号用于大多数非语音通话动词，PROXY 指的是音频使用的设备，Rx 表示输出)并指定要写入的 ALSA 控件，以及写入的内容，何时应该启用/禁用用例.最后，它列出了要在此用例中使用的 ALSA PCM 播放/捕获设备.例如，PlaybackPCM 0 表示应该使用播放设备 0(ALSA 卡被暗示为代表内置硬件编解码器的卡，通常是卡 0).这些动词由音频 HAL 根据用例(音乐播放、语音通话、录音……)、您附加的附件等选择.

This defines a verb (essentially the basis of an audio use-case; there are also modifiers that can be applied on top of verbs for stuff like simultaneous playback and recording) with the name "HiFiPROXY Rx" (the HiFi moniker is used for most non-voice-call verbs, PROXY refers to the audio device used, and Rx means output) and specifies which ALSA control(s) to write to, and what to write to them, when the use-case should be enabled / disabled. Finally it lists the ALSA PCM playback / capture devices to use in this use-case. For example, PlaybackPCM 0 means that playback device 0 should be used (the ALSA card is implied to be the one that represents the built-in hardware codec, which typically is card 0). These verbs are selected by the audio HAL based on the use-case (music playback, voice call, recording, ...), what accessories you've got attached, etc.

如果您在 soc/msm/msm-pcm-routing.c?h=kk_3.5_rb1.16#n2719" rel="noreferrer">msm_qdsp6_widgets 表在 msm-pcm-routing.c 你会看到它指的是 名为 afe_pcm_rx_mixer_controls 的混音器控件列表如下所示:

If you look up "AFE_PCM_RX Audio Mixer" in the msm_qdsp6_widgets table in msm-pcm-routing.c you'll see that it refers to a list of mixer controls named afe_pcm_rx_mixer_controls that looks like this:

static const struct snd_kcontrol_new afe_pcm_rx_mixer_controls[] = {
    SOC_SINGLE_EXT("MultiMedia1", MSM_BACKEND_DAI_AFE_PCM_RX,
    MSM_FRONTEND_DAI_MULTIMEDIA1, 1, 0, msm_routing_get_audio_mixer,
    msm_routing_put_audio_mixer),
    SOC_SINGLE_EXT("MultiMedia2", MSM_BACKEND_DAI_AFE_PCM_RX,
    ... and so on...

这列出了您可以连接到后端 DAI (AFE_PCM_RX) 的前端 DAI.要了解它们之间的关系，请参阅这些图表.
AFE_PCM_RX 和 AFE_PCM_TX 是 Qualcomm 某些平台上的一对 DAI，它们实现了一种虚拟/代理设备.您所做的是将音频输入 AFE_PCM_RX，然后由多媒体 DSP (QDSP) 处理，然后您可以通过 AFE_PCM_TX 读取它.这用于实现 USB 和 WiFi 音频路由，以及 A2DP IIRC.

This lists the front end DAIs that you are allowed to connect to the back end DAI (AFE_PCM_RX). To get an idea of how these relate to one another, see these diagrams.
AFE_PCM_RX and AFE_PCM_TX is a pair of DAIs on some of Qualcomm's platforms that implement a sort of dummy/proxy device. What you do is feed audio into AFE_PCM_RX which then gets processed by the multimedia DSP (QDSP), and then you can read it back through AFE_PCM_TX. This is used to implement USB and WiFi audio routing, and also A2DP IIRC.

回到 AFE_PCM_RX Audio Mixer MultiMedia1 行:这表示您正在将 MultiMedia1 送入 AFE_PCM_RX Audio Mixer.MultiMedia1 用于正常播放/录音，对应pcmC0D0(你应该可以用adb shell cat/proc/asound 列出你手机上的设备/设备).还有其他前端 DAI，例如 MultiMedia3 和 MultiMedia5，用于低延迟播放和低功耗音频播放等特殊情况.
当您将 MultiMedia1 输入到 AFE_PCM_RX 音频混合器 时，您写入卡 0 上播放设备 0 的所有内容都将输入 AFE_PCM_RX 后端 DAI.要回读它，您可以设置一个 UCM 动词，它执行类似 'MultiMedia1 Mixer AFE_PCM_TX':1:1 的操作，然后您可以从 pcmC0D0c(应该成为默认的 ALSA 捕获设备).

Back to the AFE_PCM_RX Audio Mixer MultiMedia1 line: This says that you're feeding MultiMedia1 into the AFE_PCM_RX Audio Mixer. MultiMedia1 is used for normal playback/recording, and corresponds to pcmC0D0 (you should be able to list the devices on your phone with adb shell cat /proc/asound/devices). There are other front end DAIs, like MultiMedia3 and MultiMedia5 that are used in special cases like low-latency playback and low-power audio playback.
When you feed MultiMedia1 to the AFE_PCM_RX Audio Mixer everything you write to playback device 0 on card 0 will be fed into the AFE_PCM_RX back end DAI. To read it back you could set up a UCM verb that does something like 'MultiMedia1 Mixer AFE_PCM_TX':1:1, and then you'd read from pcmC0D0c (which should be the default ALSA capture device).

一个简单的测试是从您的手机中提取 UCM 设置文件(应位于 /system/etc/ 下的某处)并修改 "HiFi" 动词的EnableSequence 类似于:

A simple test would be to pull the UCM settings file from your phone (should be located somewhere under /system/etc/) and amend the "HiFi" verb's EnableSequence with something like:

'AFE_PCM_RX Audio Mixer MultiMedia1':1:1
'AFE_PCM_RX Audio Mixer MultiMedia3':1:1
'AFE_PCM_RX Audio Mixer MultiMedia5':1:1

(在 DisableSequence 中类似，但在每行末尾带有 :1:0).

(and similarly in the DisableSequence, but with :1:0 at the end of each line).

然后转到 "Capture Music" 修饰符(这是正常录音的名称不佳的修饰符)并将 SLIM_0_TX 更改为 AFE_PCM_TX.

Then go to the "Capture Music" modifier (this is the poorly named modifier for normal recording) and change SLIM_0_TX to AFE_PCM_TX.

将修改后的 UCM 设置文件复制回手机(需要 root 权限)，然后重启手机.然后开始一些播放(连接有线耳机/耳机，并禁用触摸声音，以便不会选择低延迟动词)，并从 AudioSource.MIC 开始录音.之后，检查录音，看看您是否能够录制播放音频.如果没有，那么可能选择了低功率音频动词，您必须修改 "HiFi Low Power" 动词，类似于您对 "HiFi" 动词.如果您在音频 HAL 中启用了所有调试打印(即在所有可以找到它的 cpp 文件中取消注释 #define LOG_NDEBUG 0)，它将对您有所帮助，以便您可以看到哪些 UCM 动词/被选中的修饰符.

Copy your modified UCM settings file back to the phone (requires root permission), and reboot the phone. Then start some playback (have a wired headset/headphone attached, and disable touch sounds so that the low-latency verb doesn't get selected), and start a recording from AudioSource.MIC. Afterwards, check the recording and see if you were able to record the playback audio. If not, then perhaps the low-power audio verb was selected and you'll have to modify the "HiFi Low Power" verb similarly to what you did with the "HiFi" verb. It will help you if you have all the debug prints enabled in the audio HAL (i.e. uncomment #define LOG_NDEBUG 0 in all the cpp files where you can find it) so that you can see which UCM verbs / modifiers that get selected.

我上面描述的修改有点乏味，因为您必须涵盖所有相关动词和修饰符的所有 MultiMedia 前端 DAI.
IIRC，我能够将其简化为每个动词/修饰符仅一行:

The modification I described above gets a bit tedious since you have to cover all the MultiMedia front end DAIs for all relevant verbs and modifiers.
IIRC, I was able to simplify this into just a single line per verb/modifier:

'AFE_PCM_RX Port Mixer SLIM_0_RX':1:1

如果您查看 "HiFi"、"HiFi Low Power"、"HiFi Lowlatency" 动词，您会发现它们都使用 SLIMBUS_0_RX 后端 DAI，所以我通过使用 AFE_PCM_RX Port Mixer 来利用它，它让我建立从后端 DAI 到另一个的连接后端 DAI.如果您查看 msm-pcm-routing.c 中的 afe_pcm_rx_port_mixer_controls 和 intercon 表，您会注意到没有 SLIM_0_RX<AFE_PCM_RX Port Mixer 的/code> 条目，因此您必须自己添加它们(这只是复制粘贴一些现有行并更改名称的问题).

If you look at the "HiFi", "HiFi Low Power", "HiFi Lowlatency" verbs you'll see that they all use the SLIMBUS_0_RX back end DAI, so I'm taking advantage of that by using the AFE_PCM_RX Port Mixer which lets me set up a connection from a back end DAI to another back end DAI. If you look at the afe_pcm_rx_port_mixer_controls and intercon tables in msm-pcm-routing.c you'll notice that there's no SLIM_0_RX entry for AFE_PCM_RX Port Mixer, so you'd have to add those yourself (it's just a matter of copy-pasting some of the existing lines and changing the names).

您可能需要进行的其他一些更改:

Some of the other changes you'd probably have to make:

• 在 frameworks/base 和 frameworks/av(例如AudioManager、AudioService、AudioSystem)你必须添加一个新的AudioSource 保持不变，并确保在所有必要的地方都能识别它.

• In frameworks/base and frameworks/av (e.g. AudioManager, AudioService, AudioSystem) you'd have to add a new AudioSource constant and make sure that it gets recognized in all the necessary places.

在 UCM 设置文件中，您必须添加一些新的动词/修饰符，以便在使用新的 AudioSource 时正确设置 ALSA 控件.

In the UCM settings file you'd have to add some new verbs / modifiers to set up the ALSA controls properly when your new AudioSource is used.

在音频 HAL 中，您必须进行一些更改，以便在使用新的 AudioSource 时选择您的新动词/修饰符.请注意，有一个名为 AudioPolicyManagerBase 的 AudioPolicyManagerALSA 基类，您可能还需要对其进行修改(它是 位于源代码树的其他位置).

In the audio HAL you'd have to make some changes so that your new verbs / modifiers get selected when your new AudioSource is used. Note that there's a base class of AudioPolicyManagerALSA called AudioPolicyManagerBase which you also might have to modify (it's located elsewhere in the source tree).

这篇关于在 Android 中重定向音频/创建备用声音路径的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！