使用NODE.JS和html5的低延迟(50ms)视频流 [英] Low Latency (50ms) Video Streaming with NODE.JS and html5
问题描述
我正在构建FPV机器人，我想通过Web浏览器通过本地wi-fi连接对其进行控制.
I'm building a FPV robot, I want to control it with a with a webbrowser over a local wi-fi connection.
我正在使用带有Raspbian Stretch的树莓派3B +.我制造了自己的电动机控制和功率调节器帽子.
I'm using a raspberry pi 3B+ with Raspbian Stretch. I built my own motor control and power regulator hat.
经过大量研究测试，我决定将node.js用作http服务器和socket.io，以与我的机器人进行低延迟的双向通信.该堆栈可实现大约7毫秒的延迟.
After lots of research testing, I decided to use node.JS as http server and socket.io to provide a low latency bidirectional communication with my robot. This stack achieve about 7ms of latency.
我需要将低延迟视频从连接到RPI的USB摄像机流式传输到浏览器.我的目标是在10FPS时达到至少640x480分辨率，并具有50ms或更长时间的延迟.我很乐意牺牲视觉素养，以便机器人更快地做出反应.
I need to stream low latency video from an USB camera attached to the RPI to the browser. My target is to achieve at least 640x480 resolution at 10FPS with 50ms of latency or better. I'm happy sacrificing visual fedelity to get a quicker response from my robot.
如果可能的话，我想在UDP中进行流传输以提高流的可靠性.
If possible I would like to stream in UDP to improve the reliability of the stream.
如果可能的话，我想流传输现代网络浏览器可以本地解码的视频.我想使用H264编解码器和HTML5视频标签.如果没有其他选择，我可以退一步使用JavaScript播放器.
If possible I would like to stream a video that modern webbrowsers can natively decode. I'd like to use a H264 codec and the HTML5 video tag. I can fall back to use a javascript player if there is no other option.
我进行了广泛的研究，并尝试了许多工具.
I did an extensive research and tried many tools.
除其他外，我尝试了VLC，mjpg流光，gstreamer和raspivid.几次我浏览Web浏览器可以观看的视频流，但在320x240的情况下，充其量只有700毫秒的延迟.距离我的目标非常远.
Amongst other, I tried VLC, mjpg streamer, gstreamer and raspivid. A few times I got to a stream the webbrowser could view, but at best I got a latency of 700ms at 320x240. Very very far from my target.
目前，我正在研究WebRTC解决方案.
Currently I'm looking into WebRTC solutions.
我希望获得NODE.JS软件包或其他解决方案的建议，以提供UDP H264视频流，该视频流可以通过HTML5视频标签进行解码，目标延迟时间为50ms.
I'd like suggestions for NODE.JS packages or other solutions to provide a UDP H264 video stream that can be decoded by an HTML5 video tag with a target latency of 50ms.
谢谢
感谢您的回答！我将继续更新此问题，并在解决方案可行后将其发布.
Thanks for your answers! I'll keep updating this question and I'll post the solution once it works.
按一下个性框架
我尝试了一种不同的方法，将200KB 640x480 jpg单个帧通过websocket推送，得到大约190ms的延迟.通过重用对象，我可能可以做得更好，但现在暂时搁置这种尝试.
I tried a different approach by pushing individual 200KB 640x480 jpg frame through websocket and I got a latency of about 190ms. I can probably do a lot better by reusing objects but I'm putting this attempt in hold for now.
在研究WebRTC时，我发现了一个看起来很简单的堆栈.服务器端它使用V4L2作为驱动程序，FFMPEG将代码本地转换为具有TS封装的MPEG1 http流，节点js将其转换为websocket.客户端有一个JavaScript，可对MPEG1 TS流进行解码并将画布对象绘制到HTML页面中.
While researching WebRTC I found a stack that looked easy enough. Server side it uses V4L2 as driver, FFMPEG to transcode into an MPEG1 http stream with TS encapsulation locally, node js to flip the stream into a websocket. Client side there is a javascript that decode the MPEG1 TS stream and paint a canvas object into the HTML page.
它实现640x480 @ 20FPS，延迟为240mS.对于MVP来说已经足够好了，但是我会继续努力下去.答案中的代码.
It achieves 640x480@20FPS with 240mS of latency. Good enough for an MVP, but I'll keep working to get it down. Code in the answer.
推荐答案
我希望获得NODE.JS软件包或其他解决方案的建议，以提供UDP H264视频流，该视频流可以通过HTML5视频标签进行解码，目标延迟时间为50ms.
I’d like suggestions for NODE.JS packages or other solutions to provide a UDP H264 video stream that can be decoded by an HTML5 video tag with a target latency of 50ms.
在这种配置下，几乎可以肯定这是不可能的.
That’s Almost certainly not possible in that configuration.
如果您放弃视频标签要求，而仅在浏览器中使用直接的WebRTC，则可能可以缩短到150毫秒左右.
If you drop the video tag requirement, and use just straight WebRTC in the browser, you may be able to get down to about 150ms.
这篇关于使用NODE.JS和html5的低延迟(50ms)视频流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
