HTML5视频在Android Chrome上绘制为canvas2D问题 [英] HTML5 video draw into canvas2D issue on Android Chrome
问题描述
我正在开发javascript（没有jQuery或其他框架）的VR html5页面，它使用WebGL渲染一个球体，并具有渲染流式视频的纹理。
I am developing VR html5 page in javascript (no jQuery or other frameworks) that uses WebGL to render a sphere and has texture to which streamed video is rendered into.
所有这一切在iPhone 6,6 +上工作正常，但是在Android上我已经碰壁了 - 简单地说，视频没有转移到纹理（gl.texSubImage2D）。纹理保持黑色。没有抛出WebGL错误。
All that works fine on iPhone 6, 6+, however on Android I've hit the wall - simply, video is not being transferred into texture (gl.texSubImage2D). Texture remains black. No WebGL errors thrown.
所以我创建了一个没有WebGL的测试，只是尝试播放视频并将其绘制到canvas2D中，所以我至少可以验证这些帧确实是从流式视频中提取的。
So I have created a test, without WebGL, that just tries to play a video and draw it's frames into canvas2D, so I can at least verify that frames are indeed extracted from streamed video.
视频是在用户交互（触摸）上播放的，当 canplay 事件被触发时，我开始帧loop（window.requestAnimationFrame）将视频绘制到画布中（canvas.drawImage（video，0,0））
Video is played on user interaction (touch) and when canplay event is triggered, I start frame loop (window.requestAnimationFrame) to draw video into canvas (canvas.drawImage( video, 0,0 ))
视频和画布都在文档主体上可见，下一步彼此。
Both video and canvas are on the document body visible, next to each other.
结果：在桌面上它按预期工作，两个屏幕，左边是带有本机控件的视频，右边是画布。单击播放时，视频将启动，并且画布会同时刷新。在Android Chrome 48.0.2564.106上没有绘制像素 - 画布完全是空的。
Result: On desktop it works as expected, two screens, left is video with native controls, right is canvas. When I click play, video starts, and canvas gets refreshed at the same time. On Android Chrome 48.0.2564.106 no pixels drawn - canvas is totally empty.
我已经安装了Android Chrome Beta（50.0.2661.57），它可以在那里运行，但在Android Chrome上48.0.2564.106它没有。
I have installed Android Chrome Beta (50.0.2661.57) and it works there, but on Android Chrome 48.0.2564.106 it does not.
设置视频和画布的代码：
Codes to setup video and canvas:
<!doctype html>
<html>
<head>
<meta charset="utf-8" />
<meta name="format-detection" content="telephone-no" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<title>Video Canvas Test</title>
<script>
var video;
var canvas;
var ctx;
var info;
var pass = 0;
window.onload = function()
{
video = document.createElement("video");
video.oncanplay = function(){ initializeCanvas(); }
video.onerror = function(e){ console.error("video problem"); }
video.loop = true;
video.controls = "true";
video.src = "video/big-buck-bunny_trailer.webm";
video.style.width = "400px";
video.style.height = "300px";
video.style.position = "absolute";
video.style.left = "20px";
video.style.top = "20px";
video.style.backgroundColor = "#8080FF";
canvas = document.createElement("canvas");
canvas.style.backgroundColor = "#8080FF";
canvas.style.width = "400px";
canvas.style.height = "300px";
canvas.style.position = "absolute";
canvas.style.left = "420px";
canvas.style.top = "20px";
ctx = canvas.getContext("2d");
info = document.createElement("p");
info.innerHTML = window.navigator.userAgent;
info.style.position = "absolute";
info.style.width = "200px";
info.style.height = "auto";
info.style.position = "absolute";
info.style.left = "20px";
info.style.top = "320px";
document.body.appendChild(video);
document.body.appendChild(canvas);
document.body.appendChild(info);
}
function initializeCanvas()
{
console.log("Video ready to play. Init canvas.");
ctx.canvas.width = 640; // I am 100% sure this is correct video size.
ctx.canvas.height = 360;
console.log("Video size: " + video.videoWidth + "x" + video.videoHeight);
ctx.font = "30px Arial";
updateCanvas();
}
function updateCanvas()
{
pass ++;
ctx.clearRect(0,0,ctx.canvas.width,ctx.canvas.height);
ctx.drawImage(video,0,0);
ctx.fillText("Pass: " + pass,30,120);
window.requestAnimationFrame( updateCanvas );
}
</script>
</head>
<body>
</body>
</html>
任何人都可以确认Chrome 48及更早版本上的canvas.drawImage是否不接受视频作为绘图源？对于我来说这看起来不可能，因为WebGL已经支持Chrome浏览器多年的支持，并且有很多关于视频纹理的实验。
Can anyone confirm if canvas.drawImage on Chrome 48 and older cannot accept video as drawing source? This looks impossible to me considering WebGL has been in Chrome's support for ages and with plenty of experiments with video textures out there.
我尝试了其他将视频复制到WebGL的示例纹理或Canvas2D，它们也不起作用。
I have tried other samples that copy video into WebGL texture or Canvas2D and they do not work as well.
这里有什么我缺少的吗？
Is there anything I am missing here?
推荐答案
经过深入研究后，似乎：
After thorough research it appears that:
-
Canvas2D.drawImage（视频，0,0 ）自2015年7月开始被打破（至少我在Android Chrome中找到了那时的错误报告）并且有点固定 - 不固定 - 固定 - 再次 - 再次 - 不固定 - 但我可以确认它是修复了Android Chrome 49和Android Chrome 50（Beta）。将视频中的像素解码为Canvas2D的相同问题影响了将视频绘制到WebGL纹理中。
Canvas2D.drawImage( video, 0,0 ) was broken since July 2015 (at least as I found bug reports that date from then) in Android Chrome and was kinda fixed-not-fixed-fixed-again-and-again-not-fixed - but I can confirm that it is fixed in Android Chrome 49 and in Android Chrome 50 (Beta). Same issue with decoding pixels from video to be drawn into Canvas2D has affected drawing video into WebGL texture.
唯一的解决方法是：a）自定义 - 使用JavaScript中的Websocket + JPG序列流媒体制作视频流服务器（通过打开的websocket连接接收的图像）或b）JavaScript中的JPG序列流媒体（逐个下载图像并负责加载新的 - 删除旧的）。
The only workaround is to have either: a) custom-made video streaming server using Websocket + JPG sequence streamer in JavaScript (images received through opened websocket connection) or b) JPG sequence streamer in JavaScript (downloads images one by one and takes care of loading new - deleting old).
我选择了2.b并且CDN踢了它确实适用于1024x512视频大小，这是足够好的解决方案。
I went with option 2.b and with CDN kicked in it really works great for 1024x512 video size, which was good enough solution.
我首先使用http请求预加载mp3文件，然后将其加载到Audio对象中。一旦完成，JPG流媒体将启动，然后将使用声音currentTime值开始播放。它与声音100％同步！
I would first preload mp3 file using http request, then load it into Audio object. Once that's done, JPG streamer would kick in and would then commence playback using sound currentTime value. It was 100% synchronised with sound!
然而，问题在Chrome 49和Chrome 50（测试版）中得到修复，因此这可能会在2-3个月内过时。
However, issue is fixed in Chrome 49, and Chrome 50 (beta), so this likely will be obsolete in 2-3 months.
这篇关于HTML5视频在Android Chrome上绘制为canvas2D问题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
