从HTML5视频获取原始像素数据 [英] Get raw pixel data from HTML5 video

问题描述

我想从视频元素中获取原始数据（TypedArray或其他东西），并用JavaScript操作它们。

I would like to get the raw data (TypedArray or something) from video element and manipulate them with JavaScript.

目前我创建一个新的画布，然后获取图像数据。

Currently I create a new canvas, draw the video into canvas and then get the image data.

ctx.drawImage(myVideo);
var data = ctx.getImageData(0, 0, w, h).data;

它工作正常，但它会消耗一个CPU（把视频放到画布上，并且它创建了大量的垃圾（每秒大约50 MB）。有没有其他更简单的解决方案？这将是巨大的，如果我可以传递自己的缓冲区getImageData（...）。

It works fine, but it drains a CPU (putting the video to canvas and copying back from canvas) and it creates a lot of garbage (about 50 MB each second). Is there any other simpler solution? It would be great if I could pass my own buffer to getImageData(...).

BTW。使用WebGL绘制视频并从GPU加载它并不是更快（ http://jsperf.com/getting-来自视频的原始数据

BTW. drawing video with WebGL and loading it back from GPU is not any faster :( http://jsperf.com/getting-raw-data-from-video

推荐答案

请阅读

https：// www。 scirra.com/blog/76/how-to-write-low-garbage-real-time-javascript

因为你实际上没有显示任何代码，提及浏览器或给出关于视频（分辨率，FPS，编码）的信息，这是不可能告诉你的代码为什么慢或为什么它创建显示很多垃圾。与Javasrcipt的实时视频效果可能与一些分辨率约束。

Because you don't actually show any code, mention the browser or give out information about the video (resolution, FPS, encoding) it is impossible to tell why your code is slow or why it creating show much garbage. Real-time video effects with Javasrcipt are possible with some resolution constrain.

这是使用

https://developer.mozilla.org/samples/video/chroma-key/index.xhtml a>

在< canvas> 第一。但是，我相信这个操作应该是硬件加速，因为它发生在GPU内存。从GPU下载像素可以在Javascript中操作它们，然后上传回显示内存可能是最高的分辨率。

You won't get any raw access to video data without blitting a video frame on <canvas> first. However, I believe this operation should be HW accelerated as it happens all in GPU memory. Downloading pixels down from GPU manipulating them in Javascript and then uploading back to the display memory might be the slowest step for high resolution.

优化提示

• 考虑跳过框架（较低FPS中的过程效果）

• Consider skipping frames (process effects in lower FPS)

服务器端HTML5视频呈现在服务器端呈现HTML5动画

Server-side HTML5 video rendering Rendering HTML5 animation server-side?

这篇关于从HTML5视频获取原始像素数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！