是否可以直接从 OpenSL ES(适用于 Android)中的音频资产获取字节缓冲区? [英] Is it possible to get a byte buffer directly from an audio asset in OpenSL ES (for Android)?

问题描述

我想使用 OpenSL ES FileDescriptor 对象从音频资产中获取字节缓冲区，以便我可以将其重复排入 SimpleBufferQueue，而不是使用 SL 接口播放/停止/查找文件.

I would like to get a byte buffer from an audio asset using the OpenSL ES FileDescriptor object, so I can enqueue it repeatedly to a SimpleBufferQueue, instead of using SL interfaces to play/stop/seek the file.

我想直接管理样本字节的主要原因有以下三个:

There are three main reasons why I would like to manage the sample bytes directly:

1. OpenSL 使用 AudioTrack 层来播放/停止/等播放器对象.这不仅会引入不必要的开销，而且还有一些错误，并且播放器的快速启动/停止会导致很多问题.
2. 我需要直接操作字节缓冲区以获得自定义 DSP 效果.
3. 我将要播放的剪辑很小，并且可以全部加载到内存中以避免文件 I/O 开销.另外，将我自己的缓冲区放入队列可以让我通过将 0 写入输出接收器来减少延迟，并在播放时简单地切换到样本字节，而不是停止、暂停和播放 AudioTrack.

好的，所以理由完整 - 这是我尝试过的 - 我有一个 Sample 结构，它本质上包含一个输入和输出轨道，以及一个用于保存样本的字节数组.输入是我的 FileDescriptor 播放器，输出是 SimpleBufferQueue 对象.这是我的结构:

Okay, so justifications complete - here's what I've tried - I have a Sample struct which contains, essentially, an input and output track, and a byte array to hold the samples. The input is my FileDescriptor player, and the output is a SimpleBufferQueue object. Here's my struct:

typedef struct Sample_ {
    // buffer to hold all samples
    short *buffer;      
    int totalSamples;

    SLObjectItf fdPlayerObject;
    // file descriptor player interfaces
    SLPlayItf fdPlayerPlay;
    SLSeekItf fdPlayerSeek;
    SLMuteSoloItf fdPlayerMuteSolo;
    SLVolumeItf fdPlayerVolume;
    SLAndroidSimpleBufferQueueItf fdBufferQueue;

    SLObjectItf outputPlayerObject; 
    SLPlayItf outputPlayerPlay; 
    // output buffer interfaces
    SLAndroidSimpleBufferQueueItf outputBufferQueue;        
} Sample;

初始化文件播放器后fdPlayerObject，并使用

after initializing a file player fdPlayerObject, and malloc-ing memory for my byte buffer with

sample->buffer = malloc(sizeof(short)*sample->totalSamples);

我正在使用它的 BufferQueue 接口

I'm getting its BufferQueue interface with

// get the buffer queue interface
result = (*(sample->fdPlayerObject))->GetInterface(sample->fdPlayerObject, SL_IID_ANDROIDSIMPLEBUFFERQUEUE, &(sample->fdBufferQueue));

然后我实例化一个输出播放器:

// create audio player for output buffer queue
const SLInterfaceID ids1[] = {SL_IID_ANDROIDSIMPLEBUFFERQUEUE};
const SLboolean req1[] = {SL_BOOLEAN_TRUE};
result = (*engineEngine)->CreateAudioPlayer(engineEngine, &(sample->outputPlayerObject), &outputAudioSrc, &audioSnk,
                                               1, ids1, req1);

// realize the output player
result = (*(sample->outputPlayerObject))->Realize(sample->outputPlayerObject, SL_BOOLEAN_FALSE);
assert(result == SL_RESULT_SUCCESS);

// get the play interface
result = (*(sample->outputPlayerObject))->GetInterface(sample->outputPlayerObject, SL_IID_PLAY, &(sample->outputPlayerPlay));
assert(result == SL_RESULT_SUCCESS);

// get the buffer queue interface for output
result = (*(sample->outputPlayerObject))->GetInterface(sample->outputPlayerObject, SL_IID_ANDROIDSIMPLEBUFFERQUEUE,
                                                   &(sample->outputBufferQueue));
assert(result == SL_RESULT_SUCCESS);    

  // set the player's state to playing
result = (*(sample->outputPlayerPlay))->SetPlayState(sample->outputPlayerPlay, SL_PLAYSTATE_PLAYING);
assert(result == SL_RESULT_SUCCESS);

当我想播放样本时，我使用:

When I want to play the sample, I am using:

Sample *sample = &samples[sampleNum];
// THIS WORKS FOR SIMPLY PLAYING THE SAMPLE, BUT I WANT THE BUFFER DIRECTLY 
//    if (sample->fdPlayerPlay != NULL) {
//        // set the player's state to playing
//        (*(sample->fdPlayerPlay))->SetPlayState(sample->fdPlayerPlay, SL_PLAYSTATE_PLAYING);
//    }

// fill buffer with the samples from the file descriptor
(*(sample->fdBufferQueue))->Enqueue(sample->fdBufferQueue, sample->buffer,sample->totalSamples*sizeof(short));
// write the buffer to the outputBufferQueue, which is already playing
(*(sample->outputBufferQueue))->Enqueue(sample->outputBufferQueue, sample->buffer, sample->totalSamples*sizeof(short));

但是，这会导致我的应用程序冻结并关闭.这里不对劲.另外，我不希望每次都从文件描述符的 BufferQueue 中获取样本.相反，我想将它永久存储在一个字节数组中，并在我喜欢的时候将它排入输出队列.

However, this causes my app to freeze and shut down. Something is wrong here. Also, I would prefer to not get the samples from the File Descriptor's BufferQueue each time. Instead, I'd like to permanently store it in a byte array, and Enqueue that to the output whenever I like.

推荐答案

解码为 PCM 在 API 级别 14 及更高级别可用.

Decoding to PCM is available at API level 14 and higher.

当您创建解码器播放器时，您需要将 Android 简单缓冲区队列设置为数据接收器:

When you create decoder player you need set Android simple buffer queue as the data sink:

// For init use something like this:
SLDataLocator_AndroidFD locatorIn = {SL_DATALOCATOR_ANDROIDFD, decriptor, start, length};
SLDataFormat_MIME dataFormat = {SL_DATAFORMAT_MIME, NULL, SL_CONTAINERTYPE_UNSPECIFIED};
SLDataSource audioSrc = {&locatorIn, &dataFormat};

SLDataLocator_AndroidSimpleBufferQueue loc_bq = {SL_DATALOCATOR_ANDROIDSIMPLEBUFFERQUEUE, 2};
SLDataSink audioSnk = { &loc_bq, NULL };

const SLInterfaceID ids[2] = {SL_IID_PLAY, SL_IID_ANDROIDSIMPLEBUFFERQUEUE};
const SLboolean req[2] = {SL_BOOLEAN_TRUE, SL_BOOLEAN_TRUE};

SLresult result = (*engineEngine)->CreateAudioPlayer(engineEngine, &(sample->fdPlayerObject), &outputAudioSrc, &audioSnk, 2, ids1, req1);

对于解码器队列，您需要将一组空缓冲区排入 Android 简单缓冲区队列，该队列将填充 PCM 数据.

For decoder queue you need enqueue a set of empty buffers to the Android simple buffer queue, which will be filled with PCM data.

此外，您还需要向解码器队列注册一个回调处理程序，当 PCM 数据准备好时，该处理程序将被调用.回调处理程序应处理 PCM 数据，将现在为空的缓冲区重新排队，然后返回.应用程序负责跟踪解码的缓冲区；回调参数列表没有包含足够的信息来指示哪个缓冲区已填充或下一个要排队的缓冲区.

Also you need register a callback handler with the decoder queue which will be called when PCM data will be ready. The callback handler should process the PCM data, re-enqueue the now-empty buffer, and then return. The application is responsible for keeping track of decoded buffers; the callback parameter list does not include sufficient information to indicate which buffer was filled or which buffer to enqueue next.

解码为 PCM 支持暂停和初始搜索.不支持音量控制、效果、循环和播放速率.

Decode to PCM supports pause and initial seek. Volume control, effects, looping, and playback rate are not supported.

从阅读解码音频到PCM适用于 Android 的 OpenSL ES 了解更多详情.

Read Decode audio to PCM from OpenSL ES for Android for more details.

这篇关于是否可以直接从 OpenSL ES(适用于 Android)中的音频资产获取字节缓冲区?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！