从浏览器中的网络摄像头获取 ReadableStream [英] Get ReadableStream from Webcam in Browser

问题描述

我想在浏览器中将网络摄像头输入作为 ReadableStream 以管道传输到 WritableStream.我曾尝试使用 MediaRecorder API，但是当我想要一个连续流时，该流被分成单独的 blob.我认为解决方案可能是将 MediaRecorder 块通过管道传输到一个统一的缓冲区并作为连续流从中读取，但我不确定如何让该中间缓冲区工作.

I would like to get webcam input as a ReadableStream in the browser to pipe to a WritableStream. I have tried using the MediaRecorder API, but that stream is chunked into separate blobs while I would like one continuous stream. I'm thinking the solution might be to pipe the MediaRecorder chunks to a unified buffer and read from that as a continuous stream, but I'm not sure how to get that intermediate buffer working.

mediaRecorder = new MediaRecorder(stream, recorderOptions);
mediaRecorder.ondataavailable = handleDataAvailable;

mediaRecorder.start(1000);

async function handleDataAvailable(event) {
  if (event.data.size > 0) {
    const data: Blob = event.data;
    // I think I need to pipe to an intermediate stream? Not sure how tho 
    data.stream().pipeTo(writable);
  }
}

推荐答案

目前我们无法真正访问 MediaStream 的原始数据，我们最接近视频的是 MediaRecorder API，但这将对数据进行编码并通过块不是流.

Currently we can't really access the raw data of the MediaStream, the closest we have for video is the MediaRecorder API but this will encode the data and works by chunks not as a stream.

但是，有一个新的 MediaCapture Transform W3C 小组致力于 MediaStreamTrackProcessor 接口完全按照您的要求运行，并且已经在 下的 Chrome 中可用chrome://flags/#enable-experimental-web-platform-features 标志.
读取结果流时，根据您通过的轨道类型，您将获得对 的访问权限VideoFrames 或 AudioFrames 由新的 WebCodecs API.

However, there is a new MediaCapture Transform W3C group working on a MediaStreamTrackProcessor interface doing exactly what you want and which is already available in Chrome under the chrome://flags/#enable-experimental-web-platform-features flag.
When reading the resulting stream and depending on which kind of track you passed, you'll gain access to VideoFrames or AudioFrames which are being added by the new WebCodecs API.

if( window.MediaStreamTrackProcessor ) {
  const track = getCanvasTrack();
  const processor = new MediaStreamTrackProcessor( track );
  const reader = processor.readable.getReader();
  readChunk();
  function readChunk() {
    reader.read().then( ({ done, value }) => {
      // value is a VideoFrame
      // we can read the data in each of its planes into an ArrayBufferView
      const channels = value.planes.map( (plane) => {
        const arr = new Uint8Array(plane.length);
        plane.readInto(arr);
        return arr;
      });
      value.close(); // close the VideoFrame when we're done with it

      log.textContent = "planes data (15 first values):\n" +
        channels.map( (arr) => JSON.stringify( [...arr.subarray(0,15)] ) ).join("\n");
      if( !done ) {
        readChunk();
      }
    });
  }
}
else {
  console.error("your browser doesn't support this API yet");
}

function getCanvasTrack() {
  // just some noise...
  const canvas = document.getElementById("canvas");
  const ctx = canvas.getContext("2d");
  const img = new ImageData(300, 150);
  const data = new Uint32Array(img.data.buffer);
  const track = canvas.captureStream().getVideoTracks()[0];

  anim();
  
  return track;
  
  function anim() {
    for( let i=0; i<data.length;i++ ) {
      data[i] = Math.random() * 0xFFFFFF + 0xFF000000;
    }
    ctx.putImageData(img, 0, 0);
    if( track.readyState === "live" ) {
      requestAnimationFrame(anim);
    }
  }
  
}

<pre id="log"></pre>
<p>
Source<br>
<canvas id="canvas"></canvas>
</p>

这篇关于从浏览器中的网络摄像头获取 ReadableStream的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！