使用 Web Audio API 时的播放会在每个块的开头跳过 [英] Playback when using Web Audio API skips at the beginning of every chunk

问题描述

我一直在开发一个音乐应用，今天我终于开始尝试在其中播放音乐.

I've been building a music app and today I finally got around to the point where I started trying to work playing the music into it.

作为我的环境设置的概述，我将音乐文件存储为 MP3，我已使用 GridFS 将其上传到 MongoDB 数据库中.然后我使用 socket.io 服务器从 MongoDB 数据库下载块，并将它们作为单独的发射发送到前端，由 Web Audio API 处理并安排播放.

As an outline of how my environment is set up, I am storing the music files as MP3s which I have uploaded into a MongoDB database using GridFS. I then use a socket.io server to download the chunks from the MongoDB database and send them as individual emits to the front end where the are processed by the Web Audio API and scheduled to play.

当它们播放时，它们都按正确的顺序排列，但是每次(大概是在块之间)都存在我似乎无法摆脱的非常微小的故障或跳过相同的位置.据我所知，它们都安排在彼此相邻的位置，所以我找不到它们之间应该存在任何差距或重叠的原因.任何帮助，将不胜感激.代码如下:

When they play, they are all in the correct order but there is this very tiny glitch or skip at the same spots every time (presumably between chunks) that I can't seem to get rid of. As far as I can tell, they are all scheduled right up next to each other so I can't find a reason why there should be any sort of gap or overlap between them. Any help would be appreciated. Here's the code:

套接字路由

socket.on('stream-audio', () => {
  db.client.db("dev").collection('music.files').findOne({"metadata.songId": "3"}).then((result) =>{
      const bucket = new GridFSBucket(db.client.db("dev"), {
        bucketName: "music"
      });
      bucket.openDownloadStream(result._id).on('data',(chunk) => {
      socket.emit('audio-chunk',chunk)
    });
  });
});

前端

//These variable are declared as object variables, hence all of the "this" keywords
context: new (window.AudioContext || window.webkitAudioContext)(),
freeTime: null,
numChunks: 0,
chunkTracker: [],

...

this.socket.on('audio-chunk', (chunk) => {
      //Keeping track of chunk decoding status so that they don't get scheduled out of order
      const chunkId = this.numChunks
      this.chunkTracker.push({
        id: chunkId,
        complete: false,
      });
      this.numChunks += 1;

      //Callback to the decodeAudioData function
      const decodeCallback = (buffer) => {
        var shouldExecute = false;
        const trackIndex = this.chunkTracker.map((e) => e.id).indexOf(chunkId);

        //Checking if either it's the first chunk or the previous chunk has completed
        if(trackIndex !== 0){
          const prevChunk = this.chunkTracker.filter((e) => e.id === (chunkId-1))
          if (prevChunk[0].complete) {
            shouldExecute = true;
          }
        } else {
          shouldExecute = true;
        }

        //THIS IS THE ACTUAL WEB AUDIO API STUFF
        if (shouldExecute) {
          if (this.freeTime === null) {
            this.freeTime = this.context.currentTime
          }
          const source = this.context.createBufferSource();
          source.buffer = buffer
          source.connect(this.context.destination)
          if (this.context.currentTime >= this.freeTime){
            source.start()
            this.freeTime = this.context.currentTime + buffer.duration
          } else {
            source.start(this.freeTime)
            this.freeTime += buffer.duration
          }
          //Update the tracker of the chunks that this one is complete
          this.chunkTracker[trackIndex] = {id: chunkId, complete: true}
        } else {
          //If the previous chunk hasn't processed yet, check again in 50ms
          setTimeout((passBuffer) => {
            decodeCallback(passBuffer)
          },50,buffer);
        }
      }
      decodeCallback.bind(this);
      
      this.context.decodeAudioData(chunk,decodeCallback);
    });

任何帮助将不胜感激，谢谢！

Any help would be appreciated, thanks!

推荐答案

作为我的环境设置的概述，我将音乐文件存储为 MP3，我使用 GridFS 将其上传到 MongoDB 数据库中.

As an outline of how my environment is set up, I am storing the music files as MP3s which I have uploaded into a MongoDB database using GridFS.

如果你愿意，你可以这样做，但现在我们有像 Minio 这样的工具，可以使用更常见的 API 使这变得更容易.

You can do this if you want, but these days we have tools like Minio, which can make this easier using more common APIs.

然后我使用 socket.io 服务器从 MongoDB 数据库下载块并将它们作为单独的发送发送到前端

I then use a socket.io server to download the chunks from the MongoDB database and send them as individual emits to the front end

不要走这条路.Web 套接字或 Socket.IO 的开销没有理由.普通的 HTTP 请求就可以了.

Don't go this route. There's no reason for the overhead of web sockets, or Socket.IO. A normal HTTP request would be fine.

由 Web Audio API 处理并安排播放的位置.

where the are processed by the Web Audio API and scheduled to play.

您不能以这种方式进行流式传输.Web Audio API 不支持有用的流媒体，除非您碰巧有原始 PCM 块，而您没有.

You can't stream this way. The Web Audio API doesn't support useful streaming, unless you happened to have raw PCM chunks, which you don't.

据我所知，它们都安排在彼此相邻的位置，所以我找不到它们之间应该存在任何差距或重叠的原因.

As far as I can tell, they are all scheduled right up next to each other so I can't find a reason why there should be any sort of gap or overlap between them.

有损编解码器不会为您提供样本准确的输出.特别是对于 MP3，如果你给它一些任意数量的样本，你最终会得到至少一个完整的 MP3 帧(~576 个样本)输出.现实情况是，您需要在第一个音频帧之前的数据才能正常工作.如果要解码流，则需要从流开始.您无法通过这种方式独立解码 MP3.

Lossy codecs aren't going to give you sample-accurate output. Especially with MP3, if you give it some arbitrary number of samples, you're going to end up with at least one full MP3 frame (~576 samples) output. The reality is that you need data ahead of the first audio frame for it to work properly. If you want to decode a stream, you need a stream to start with. You can't independently decode MP3 this way.

幸运的是，该解决方案还简化了您的工作.只需从您的服务器返回一个 HTTP 流，并使用 HTML 音频元素 或 new Audio(url).浏览器将处理所有缓冲.只要确保您的服务器处理范围请求，就可以了.

Fortunately, the solution also simplifies what you're doing. Simply return an HTTP stream from your server, and use an HTML audio element <audio> or new Audio(url). The browser will handle all the buffering. Just make sure your server handles range requests, and you're good to go.

这篇关于使用 Web Audio API 时的播放会在每个块的开头跳过的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！