与视频同步播放多个音轨之一 [英] Playing one of multiple audio tracks in sync with a video

问题描述

我正在尝试在网络浏览器中播放视频，原始视频带有两个或多个音频流，每种音频流使用不同的语言.我想让用户可以选择切换他们正在收听的音轨.

I'm trying to play a video in a web browser, the original video comes with two or more audio streams, each in a different language. I want to give the user the option to switch which audio track they're listening to.

我尝试使用 audioTracks 上的视频元素，但是尽管说大多数浏览器都支持该标志，至少在Firefox和Chrome中我不说它是完全有效的(在Firefox中，它仅显示第一首曲目，并且元数据有误，并且在Chrome中，只要您将主音轨静音，视频就会暂停，并且您必须搜索视频才能使它真正继续播放).

I tried using audioTracks on the video element, but despite saying it's supported behind a flag in most browsers, at least in Firefox and Chrome I wouldn't say it's working at all (in Firefox it only shows the first track and the metadata was wrong, and in Chrome the video would pause as soon as you muted the main track, and you had to seek the video to get it to actually continue playing).

我尝试使用 ffmpeg 分别保存各个音轨，并尝试与视频同步播放(设置 audio.currentTime = video.currentTime 以响应一些视频上的事件，例如 play ， playing ， pause ， seeked ， stalled )，使用

I tried using ffmpeg to save the individual audio tracks separately and tried playing them in sync with the video (setting audio.currentTime = video.currentTime in response to several events on the video like play, playing, pause, seeked, stalled), playing both audio tracks in <audio> elements connected to GainNodes using the Web Audio API (switching audio tracks sets the gain to 1 for the track you want and 0 for the rest). This seems to be working flawlessly in Chrome, but Firefox is all over the place and even after syncing the currentTime properties the actual audio is off by a second or more.

我看到其他人抱怨类似的问题，因为MP3的定时关闭，但是我正在使用AAC.在这些情况下，解决方案是不为音频使用可变比特率，但似乎并没有改善它( ffmpeg -i video.mkv -map 0:a:0 -acodec aac -b:a 128k track-0.aac )

I saw other people complaining about similar issues with the timing being off with MP3, but I'm using AAC. The solution in those cases was to not use variable bitrates for the audio but that didn't seem to improve it (ffmpeg -i video.mkv -map 0:a:0 -acodec aac -b:a 128k track-0.aac)

有什么好的策略可以做到这一点吗?如果可以避免的话，我宁愿每个音轨都没有重复的视频文件.

Is there any good strategy for doing this? I'd rather not have to have duplicate video files for each audio track if I can avoid it.

推荐答案

您的最佳选择可能是使用

The best in your case is probably to use the Media Source Extension (MSE) API.
This will allow you to switch only the audio source while keeping playing the original video.
Since we will replace the whole audio SourceBuffer's content with the other audio source, we won't have sync issues, for the player, it will be just as if there was a single audio source.

(async() => {
  const vid = document.querySelector( "video" );
  const check = document.querySelector( "input" );
  // video track as ArrayBuffer
  const bufvid = await getFileBuffer( "525d5ltprednwh1/test.webm" );
  // audio track one
  const buf300 = await getFileBuffer( "p56kvhwku7pdzd9/beep300hz.webm" );
  // audio track two
  const buf800 = await getFileBuffer( "me3y69ekxyxabhi/beep800hz.webm" );
  
  const source = new MediaSource();
  // load our MediaSource into the video
  vid.src = URL.createObjectURL( source );
  // when the MediaSource becomes open
  await waitForEvent( source, "sourceopen" );

  // append video track
  const vid_buffer = source.addSourceBuffer( "video/webm;codecs=vp8" );
  vid_buffer.appendBuffer( bufvid );

  // append one of the audio tracks
  const aud_buffer =  source.addSourceBuffer( "audio/webm;codecs=opus" );
  aud_buffer.appendBuffer( check.checked ? buf300 : buf800 );
  // wait for both SourceBuffers to be ready
  await Promise.all( [
    waitForEvent( aud_buffer, "updateend" ),
    waitForEvent( vid_buffer, "updateend" )
  ] );
  // Tell the UI the stream is ended (so that 'ended' can fire)
  source.endOfStream();
  
  check.onchange = async (evt) => {
    // remove all the data we had in the Audio track's buffer
    aud_buffer.remove( 0, source.duration );
    // it is async, so we need to wait it's done
    await waitForEvent( aud_buffer, "updateend" );
    // no we append the data of the other track
    aud_buffer.appendBuffer( check.checked ? buf300 : buf800 );
    // also async
    await waitForEvent( aud_buffer, "updateend" );
    // for ended to fire
    source.endOfStream();
  };

})();

// helpers
function getFileBuffer( filename ) {
  return fetch( "https://dl.dropboxusercontent.com/s/" + filename )
    .then( (resp) => resp.arrayBuffer() );
}
function waitForEvent( target, event ) {
  return new Promise( res => {
    target.addEventListener( event, res, { once: true } );
  } );
}

video { max-width: 100%; max-height: 100% }

<label>Use 300Hz audio track instead of 800Hz <input type="checkbox"></label><br>
<video controls></video>

这篇关于与视频同步播放多个音轨之一的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！