在 Node.js 上播放来自 Web Audio API 的 PCM 流 [英] Playing PCM stream from Web Audio API on Node.js

问题描述

我正在使用网络音频 API 从浏览器流式传输录制的 PCM 音频.

I'm streaming recorded PCM audio from a browser with web audio api.

我正在使用 binaryJS(websocket 连接) 将它流式传输到 nodejs 服务器，我正在尝试播放该流服务器使用扬声器 npm 模块.

I'm streaming it with binaryJS (websocket connection) to a nodejs server and I'm trying to play that stream on the server using the speaker npm module.

这是我的客户.音频缓冲区最初是非交错的 IEEE 32 位线性 PCM，标称范围在 -1 之间和 +1.我从两个 PCM 频道中的一个开始，然后在下面播放.

This is my client. The audio buffers are at first non-interleaved IEEE 32-bit linear PCM with a nominal range between -1 and +1. I take one of the two PCM channels to start off and stream it below.

var client = new BinaryClient('ws://localhost:9000');
var Stream = client.send();

recorder.onaudioprocess = function(AudioBuffer){
    var leftChannel = AudioBuffer.inputBuffer.getChannelData (0);
    Stream.write(leftChannel);
}

现在我将数据作为缓冲区接收并尝试将其从 npm 包写入扬声器对象.

Now I receive the data as a buffer and try writing it to a speaker object from the npm package.

var Speaker = require('speaker');

var speaker = new Speaker({
  channels: 1,          // 1 channel
  bitDepth: 32,         // 32-bit samples
  sampleRate: 48000,     // 48,000 Hz sample rate
  signed:true
});

server.on('connection', function(client){
    client.on('stream', function(stream, meta){
        stream.on('data', function(data){
            speaker.write(leftchannel);
        });
    });
});

结果是我的笔记本电脑的扬声器发出了尖锐的尖叫声，这显然不是记录的内容.这也不是反馈.我可以确认客户端上的录制缓冲区是有效的，因为我尝试将它们写入 WAV 文件并且播放正常.

The result is a high pitch screech on my laptop's speakers, which is clearly not what's being recorded. It's not feedback either. I can confirm that the recording buffers on the client are valid since I tried writing them to a WAV file and it played back fine.

演讲者文档和有关 AudioBuffer 的文档

我已经被这个问题难住了好几天.有人能找出问题所在，或者提供不同的方法吗?

I've been stumped on this for days. Can someone figure out what is wrong or perhaps offer a different approach?

首先，我错误地使用了 websocket API.我更新了上面以正确使用它.

First off, I was using the websocket API incorrectly. I updated above to use it correctly.

我需要将音频缓冲区转换为整数数组缓冲区.我选择使用 Int16Array.由于给定的音频缓冲区的范围在 1 到 -1 之间，因此只需乘以新的 ArrayBuffer 范围(32767 到 -32768)即可.

I needed to convert the audio buffers to an array buffer of integers. I choose to use Int16Array. Since the given audio buffer has a range in-between 1 and -1, it was as simple as multiplying by the range of the new ArrayBuffer (32767 to -32768).

recorder.onaudioprocess = function(AudioBuffer){

    var left = AudioBuffer.inputBuffer.getChannelData (0);

    var l = left.length;
    var buf = new Int16Array(l)

    while (l--) {
        buf[l] = left[l]*0xFFFF;    //convert to 16 bit
    }

    Stream.write(buf.buffer);

}

推荐答案

看起来您正在将流作为 meta 对象发送.

It looks like you're sending your stream through as the meta object.

根据文档，BinaryClient.send 以该顺序接收一个 data 对象(流)和一个 meta 对象.stream<的回调/a> 事件在第一个参数中接收流(作为 BinaryStream 对象，而不是 Buffer)，在第二个参数中接收 meta 对象.

According to the docs, BinaryClient.send takes a data object (the stream) and a meta object, in that order. The callback for the stream event receives the stream (as a BinaryStream object, not a Buffer) in the first parameter and the meta object in the second.

您正在传递 send() 字符串 'channel' 作为流和来自 getChannelData()<的 Float32Array/code> 作为元对象.也许如果您要交换这两个参数(或仅使用 client.send(leftChannel))，然后更改服务器代码以将 stream 传递给 speaker.write 而不是 leftchannel(它可能应该重命名为 meta，或者如果你不需要它就删除它)，它可能会工作.

You're passing send() the string 'channel' as the stream and the Float32Array from getChannelData() as the meta object. Perhaps if you were to swap those two parameters (or just use client.send(leftChannel)) and then change the server code to pass stream to speaker.write instead of leftchannel (which should probably be renamed to meta, or dropped if you don't need it), it might work.

请注意，由于 Float32Array 不是流或缓冲区对象，BinaryJS 可能 尝试将其分块发送.您可能想要发送 leftChannel.buffer(ArrayBuffer 在那个对象后面) 代替.

Note that since Float32Array isn't a stream or buffer object, BinaryJS might try to send it in one chunk. You may want to send leftChannel.buffer (the ArrayBuffer behind that object) instead.

让我知道这是否适合您；我现在无法测试您的确切设置.

Let me know if this works for you; I'm not able to test your exact setup right now.

这篇关于在 Node.js 上播放来自 Web Audio API 的 PCM 流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！