使用画布编辑视频时如何减少getImageData的内存使用量? [英] How to reduce getImageData's memory usage when using canvas to edit video?
问题描述
我一直在尝试使用画布来编辑视频，方法是在屏幕外画布上绘图，然后使用 getImageData 做一些工作，然后将其放到我的屏幕画布上.它可以工作，但即使是 480x360 的小视频，Chrome 内存使用量也会不断增加，直到崩溃(在我的机器上大约需要 10 分钟，较大的视频更少).这个问题在 Firefox 中似乎更好，但仍然受到大量内存使用的影响.
I've been trying to use a canvas to edit video by drawing to a offscreen canvas and then using getImageData to do some work and then putting that onto my onscreen canvas. It works, but even with a small 480x360 video, Chrome memory usage keeps increasing until it crashes (takes about ten minutes on my machine, less with larger video). The issue seems to be better in Firefox, but still suffers from large memory usage.
我意识到每个 getImageData 调用需要大约 3MB 的内存，但即便如此，我还是觉得应该有办法让 Chrome 使用少于 1GB 的内存.我降低了帧率，这有助于但不能解决问题.我能做些什么来确保更及时地释放 imageData 内存?
I realize that each getImageData call takes ~3MB of memory, but even then I feel like there should be a way to get Chrome to use less than 1GB of memory. I've lowered the framerate, which helps but does not solve the issue. Is there anything I can do to ensure that the imageData memory is freed in a more timely fashion?
<!DOCTYPE html>
<meta charset='utf-8'>
<html>
<head>
<title>Video Test</title>
</head>
<body>
<canvas id='display'>
</canvas>
<script type='text/javascript'>
var canvas = document.getElementById('display');
var context = canvas.getContext('2d');
var bCanvas = document.createElement('canvas');
var bContext = bCanvas.getContext('2d');
var video = document.createElement('video');
video.src = 'VIDEO HERE';
video.autoplay = true;
video.loop = true;
var last;
var interval = 35;
video.addEventListener('play', function() {
canvas.width = video.videoWidth;
canvas.height = video.videoHeight;
bCanvas.width = video.videoWidth;
bCanvas.height = video.videoHeight;
last = performance.now();
window.requestAnimationFrame(draw);
}, false);
function draw(time) {
if(time - last > interval) {
bContext.drawImage(video,0,0,bCanvas.width,bCanvas.height);
var imageData = bContext.getImageData(0,0,bCanvas.width,bCanvas.height);
context.putImageData(imageData,0,0);
last = time;
}
window.requestAnimationFrame(draw);
}
</script>
</body>
</html>
推荐答案
由于您要实现的是色度键效果，因此您可以在下采样帧上进行色度检测，在其上设置透明度然后重新绘制它在输出画布上以正常比例显示.
Since what you are trying to achieve is a chroma key effect, you could do the chroma detection on a down-sampled frame, set the transparency on it then redraw it at a normal scale on the output canvas.
然后，由于您的输出上下文的 globalCompositeOperation 属性设置为 "destination-in"，您可以只绘制原始框架的非透明部分，保持其原始质量:
Then, thanks to the globalCompositeOperation property of your output context set to "destination-in", you can draw only the non-transparent part of your original frame, keeping its original quality :
// Define our canvases
var output = document.createElement('canvas'),
ctx = output.getContext('2d'),
buffer = output.cloneNode(),
buf = buffer.getContext('2d');
document.body.appendChild(output);
var threshold = colorThreshold.value,
color = hexToRgb(colorInp.value.split('#')[1]);
var lastCall = 0;
function draw() {
requestAnimationFrame(draw);
// if the video is still at the same frame, we don't need to process anything
if (video.currentTime === lastCall)
return;
// video.pause();
lastCall = video.currentTime;
// clear our output canvas
ctx.clearRect(0, 0, output.width, output.height);
ctx.drawImage(video, 0, 0, output.width, output.height);
// draw a downsampled frame on the buffer canvas
buf.drawImage(video, 0, 0, buffer.width, buffer.height);
// get this downsampled canvas's imageData
var image = buf.getImageData(0, 0, buffer.width, buffer.height),
data = image.data;
var t = threshold / 2;
// loop through the imageData pixels
for (var i = 0; i < data.length; i += 4) {
// for a correct Chroma key, this should be improved
if ((color[0] - t) <= data[i] && data[i] <= (color[0] + t) &&
(color[1] - t) <= data[i + 1] && data[i + 1] <= (color[1] + t) &&
(color[2] - t) <= data[i + 2] && data[i + 2] <= (color[2] + t)) {
// set the alpha channel to 0
data[i + 3] = 0;
}
}
// redraw our now-tranparent image on the buffer
buf.putImageData(image, 0, 0);
// set our context's gCO to destination-in ...
ctx.globalCompositeOperation = 'destination-in';
// resample the buffer to a normal scale (bad quality)
ctx.drawImage(buffer, 0, 0, output.width, output.height);
// reset the context's gCO
ctx.globalCompositeOperation = 'source-over';
}
colorThreshold.addEventListener('input', function() {
threshold = this.value;
});
colorInp.addEventListener('input', function() {
color = hexToRgb(this.value.split('#')[1]);
});
cutQ.addEventListener('input', function() {
buffer.width = (output.width / 100) * this.value;
buffer.height = (output.height / 100) * this.value;
});
video.addEventListener('loadedmetadata', function() {
output.width = this.videoWidth;
output.height = this.videoHeight;
buffer.width = (output.width / 100) * cutQ.value;
buffer.height = (output.height / 100) * cutQ.value;
draw();
});
// convert our input's value to rgb
function hexToRgb(hex) {
var bigint = parseInt(hex, 16),
r = (bigint >> 16) & 255,
g = (bigint >> 8) & 255,
b = bigint & 255;
return [r, g, b];
}canvas {
width:100%;
border: 1px solid;
background: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAoAAAAKCAYAAACNMs+9AAAAGUlEQVQYlWP4z8DwHx0zYANDQeEgcw5FCgHKJ2OdLLDjkAAAAABJRU5ErkJggg=='), repeat;
}Color to pick :
<input type="color" value="#30f062" id="colorInp" /><br>
Cut quality :
<input type="range" min="10" max="100" step=".5" value="80" id="cutQ" /><br>
Color Threshold :
<input type="range" min="0" max="255" step=".5" value="166" id="colorThreshold" /><br>
<video id="video" style="position:absolute;visibility:hidden;z-index:-1" autoplay="" muted="true" loop="true" src="https://dl.dropboxusercontent.com/s/1jp0f76yvzuucj7/L0ckergn0me-PixieGreenScreen446_512kb.mp4" crossorigin="anonymous"><!--CC BY-NC-SA 2.0 http://chris.pirillo.com/--></video>这篇关于使用画布编辑视频时如何减少getImageData的内存使用量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
