CanvasCaptureMediaStream / MediaRecorder帧同步 [英] CanvasCaptureMediaStream / MediaRecorder Frame Synchronization
问题描述
使用 CanvasCaptureMediaStream 和MediaRecorder时，是否有在每一帧上获得一个事件的方式？
我需要的东西与 requestAnimationFrame（） ，但我需要它用于CanvasCaptureMediaStream（和/或MediaRecorder） ）而不是窗口。 MediaRecorder可能以与窗口不同的帧速率运行（可能以不常规的可分率，例如25 FPS对60 FPS），所以我想以帧率而不是窗口更新画布。
此示例目前仅适用于FireFox ，因为chrome只会停止画布流标签模糊... （可能与相关错误，但是，我的计时器似乎正在工作，但不是录音......）
：它现在实际上只适用于chrome，因为它们已修复这个错误，但由于这在FF中不再存在一个（由...引起） e10s）。
MediaStream上似乎没有任何事件让你知道帧何时被渲染到它，也不在MediaRecorder上。
即使是MediaStream的 currentTime 属性（目前仅在FF中可用）也没有似乎在使用 captureStream（）方法中传递的fps参数进行相应更改。
但你是什么似乎想要的是一个可靠的计时器，当当前标签没有聚焦时（rAF会发生），它不会失去频率。
幸运的是，WebAudio API也有一个高精度计时器，基于硬件时钟，而非屏幕刷新我们可以使用另一种定时循环，即使标签模糊也能保持其频率。
/ *一个替代的定时循环，基于AudioContext的时钟@arg回调：一个回调函数，audioContext的currentTime作为唯一参数传递@arg frequency：float in ms; @returns：一个停止函数* / function audioTimerLoop（callback，frequency）{// AudioContext时间参数以秒为单位var freq = frequency / 1000; var aCtx = new AudioContext（）; // Chrome需要将我们的振荡器节点连接到目的地//所以我们创建一个静默增益节点var silence = aCtx.createGain（）; silence.gain.value = 0; silence.connect（aCtx.destination）; onOSCend（）; var stopped = false; function onOSCend（）{osc = aCtx.createOscillator（）; osc.onended = onOSCend; osc.connect（沉默）; osc.start（0）; osc.stop（aCtx.currentTime + freq）;回调（aCtx.currentTime）; if（已停止）{osc.onended = function（）{return; }; }}; //返回一个函数来停止循环返回函数（）{stopped = true;函数start（）{//启动循环@ 25fps var stopAnim = audioTimerLoop（anim，1000/25）; //最大流速设置为25 fps cStream = canvas.captureStream（25）; let chunks = []; var recorder = new MediaRecorder（cStream）; recorder.ondataavailable = e => chunks.push（e.data）; recorder.onstop = e => {//我们可以停止循环stopAnim（）; var url = URL.createObjectURL（new Blob（chunks））; var v = document.createElement（'video'）; v.src = url; v.controls = true; document.body.appendChild（V）; } recorder.start（）; //在20秒内停止录音机，尝试在此时间内更改制表符setTimeout（function（）{recorder.stop（）;}，20000）} //在canvasvar ctx = canvas.getContext（'2d'）上移动; var x = 0; function anim（）{x =（x + 2）％（canvas.width + 100）; ctx.fillStyle ='ivory'; ctx.fillRect（0,0，canvas.width，canvas.height）; ctx.fillStyle ='red'; ctx.fillRect（x - 50,20,50,50）}; start（）;
< canvas id =canvaswidth =500height =200>< / canvas>
Nota Bene ：
在本例中，我将频率设置为25fps，但我们可以将它设置为60fps，它甚至可以在我的旧笔记本上正常工作，至少可以使用这么简单的动画。
When using CanvasCaptureMediaStream and MediaRecorder, is there a way to get an event on each frame?
What I need is not unlike requestAnimationFrame(), but I need it for the CanvasCaptureMediaStream (and/or the MediaRecorder) and not the window. The MediaRecorder could be running at a different frame rate than the window (possibly at a not regularly divisible rate, such as 25 FPS vs 60 FPS), so I want to update the canvas at its frame rate rather than the window's.
This example currently only fully works on FireFox, since chrome simply stops the canvas stream when the tab is blurred... (probably related to this bug, but well, my timer seems to be working but not the recording...)
[Edit]: it actually now works only in chrome, since they have fixed this bug, but not anymore in FF because of this one (caused by e10s).
There doesn't seem to be any event on MediaStream letting you know when a frame has been rendered to it, neither on the MediaRecorder.
Even the currentTime property of the MediaStream (currently only available in FF) doesn't seem to be changing accordingly with the fps argument passed in the captureStream() method.
But what you seem to want is a reliable timer, that won't loose its frequency when i.e the current tab is not focused (which happens for rAF).
Fortunately, the WebAudio API does also have an high precision timer, based on hardware clock, rather than on screen refresh rate.
So we can come with an alternative timed loop, able to keep its frequency even when the tab is blurred.
/*
An alternative timing loop, based on AudioContext's clock
@arg callback : a callback function
with the audioContext's currentTime passed as unique argument
@arg frequency : float in ms;
@returns : a stop function
*/
function audioTimerLoop(callback, frequency) {
// AudioContext time parameters are in seconds
var freq = frequency / 1000;
var aCtx = new AudioContext();
// Chrome needs our oscillator node to be attached to the destination
// So we create a silent Gain Node
var silence = aCtx.createGain();
silence.gain.value = 0;
silence.connect(aCtx.destination);
onOSCend();
var stopped = false;
function onOSCend() {
osc = aCtx.createOscillator();
osc.onended = onOSCend;
osc.connect(silence);
osc.start(0);
osc.stop(aCtx.currentTime + freq);
callback(aCtx.currentTime);
if (stopped) {
osc.onended = function() {
return;
};
}
};
// return a function to stop our loop
return function() {
stopped = true;
};
}
function start() {
// start our loop @25fps
var stopAnim = audioTimerLoop(anim, 1000 / 25);
// maximum stream rate set as 25 fps
cStream = canvas.captureStream(25);
let chunks = [];
var recorder = new MediaRecorder(cStream);
recorder.ondataavailable = e => chunks.push(e.data);
recorder.onstop = e => {
// we can stop our loop
stopAnim();
var url = URL.createObjectURL(new Blob(chunks));
var v = document.createElement('video');
v.src = url;
v.controls = true;
document.body.appendChild(v);
}
recorder.start();
// stops the recorder in 20s, try to change tab during this time
setTimeout(function() {
recorder.stop();
}, 20000)
}
// make something move on the canvas
var ctx = canvas.getContext('2d');
var x = 0;
function anim() {
x = (x + 2) % (canvas.width + 100);
ctx.fillStyle = 'ivory';
ctx.fillRect(0, 0, canvas.width, canvas.height);
ctx.fillStyle = 'red';
ctx.fillRect(x - 50, 20, 50, 50)
};
start();
<canvas id="canvas" width="500" height="200"></canvas>
Nota Bene :
In this example, I set the frequency to 25fps, but we can set it to 60fps and it seems to work correctly even on my old notebook, at least with such a simple animation.
这篇关于CanvasCaptureMediaStream / MediaRecorder帧同步的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
