nodejs ffmpeg在特定时间播放视频并将其流式传输到客户端 [英] nodejs ffmpeg play video at specific time and stream it to client
问题描述
我正在尝试使用nodeJS和ffmpeg创建一个基本的在线视频编辑器.
I'm trying to make a basic online video editor with nodeJS and ffmpeg.
为此，我需要2个步骤:
To do this I need 2 steps:
-
设置来自客户端的视频的出入时间，这要求客户端在特定时间观看视频，并切换视频的位置.意思是，如果将单个视频用作输入，并将其分割成较小的部分，则需要从下一个已编辑片段的开始时间开始重放.
set the in-and-out times of the videos from the client, which requires the client to view the video at specific times, and switch the position of the video. Meaning, if a single video is used as an input, and split it into smaller parts, it needs to replay from the starting time of the next edited segment, if that makes sense.
将输入输出数据发送到nodejs并使用ffmpeg将其导出为最终视频.
send the input-output data to nodejs and export it with ffmpeg as a finished vide.
首先，我想只在客户端上做1.然后将源视频上传到nodeJS，并使用ffmpeg生成相同的结果，然后将结果发送回去.
At first I wanted to do 1. purely on the client, then upload the source video(s) to nodeJS, and generate the same result with ffmpeg, and send back the result.
但是目前在客户端使用HTML进行视频处理可能会出现问题，所以现在我有一个计划变更:在nodeJS服务器上进行所有处理，包括视频播放.
But there are may problems with video processing on the client side in HTML at the moment, so now I have a change of plans: to do all of the processing on the nodeJS server, including the video playing.
这是我现在停留的部分.我知道ffmpeg与nodeJS可以以多种不同的方式使用，但是我还没有找到在特定时间戳下使用ffmpeg实时播放.mp4 webm视频并发送流式视频的方法(再次，在特定的时间戳记下).
This is the part I am stuck at now. I'm aware that ffmpeg can be used in many different ways from nodeJS, but I have not found a way to play a .mp4 webm video in realtime with ffmpeg, at a specific timestamp, and send the streaming video (again, at a certain timestamp) to the client.
我已经从ffmpeg看到了pipe:1属性，但是我找不到任何教程来使它与mp4 webm视频一起使用，以及以某种方式使用nodejs解析stdout数据并将其发送到客户端.即使我能使这部分工作，我仍然不知道要在某个时间戳实时播放视频.
I've seen the pipe:1 attribute from ffmpeg, but I couldn't find any tutorials to get it working with an mp4 webm video, and to parse the stdout data somehow with nodejs and send it to the client. And even if I could get that part to work, I still have no idea to play the video, in realtime, at a certain timestamp.
我也看过ffplay，但据我所知仅用于测试.我还没有看到通过nodejs实时获取视频数据的任何方法.
I've also seen ffplay, but that's only for testing as far as I know; I haven't seen any way of getting the video data from it in realtime with nodejs.
所以:
如何在特定时间(最好使用ffmpeg)在nodeJS中播放视频，并将其实时发送回客户端?
我已经看到的:
node.js将ffmpeg stdout实时流式传输到res a>
FFmpeg实时流式传输通过Nodejs将webm视频发送到多个http客户端
http://www.mobiuso.com/blog/2018/04/18/video-processing-with-node-ffmpeg-and-gearman/
如何获得具体的开始&Node JS在ffmpeg中结束时间?
实时流式传输:为实时低延迟配置的node-media-server + Dash.js
https://medium.com/@brianshaler/on-the-fly-video-rendering-with-node-js-and-ffmpeg-165590314f2
node.js将ffmpeg stdout实时流式传输到res
推荐答案
这个问题有点广泛，但是我已经建立了类似的东西，并将尝试为您分块回答:
This question is a bit broad, but I've built similar things and will try to answer this in pieces for you:
- 设置来自客户端的视频的出入时间，这要求客户端在特定时间观看视频，并切换视频的位置.意思是，如果将单个视频用作输入，并将其分割成较小的部分，则需要从下一个已编辑片段的开始时间开始重放.
客户端，在回放时，您可以简单地使用引用相同URL的多个HTMLVideoElement实例.
Client-side, when you play back, you can simply use multiple HTMLVideoElement instances that reference the same URL.
对于计时，您可以使用 .currentTime 属性自己进行管理.但是，您会发现JavaScript时间安排不是完美的.如果您知道实例化时的起点/终点，则可以使用媒体片段URI :
For the timing, you can manage this yourself using the .currentTime property. However, you'll find that your JavaScript timing isn't going to be perfect. If you know your start/end points at the time of instantiation, you can use Media Fragment URIs:
video.src = 'https://example.com/video.webm#t=5.5,30';
在此示例中，视频以5.5秒开始，并以30秒停止.您可以使用 ended 事件知道何时开始播放下一个剪辑.不能保证完全精确到帧，但是对于诸如实时预览之类的东西来说是相当不错的.
In this example, the video starts at 5.5 seconds, and stops at 30 seconds. You can use the ended event to know when to start playing the next clip. This isn't guaranteed to be perfectly frame-accurate, but is pretty good for something like a live preview.
但是目前在客户端使用HTML进行视频处理可能存在问题，所以现在我有一个计划变更:在nodeJS服务器上进行所有处理，...
But there are may problems with video processing on the client side in HTML at the moment, so now I have a change of plans: to do all of the processing on the nodeJS server,...
如果一致性很重要，这不是一个坏计划.
Not a bad plan, if consistency is important.
...包括正在播放的视频.
... including the video playing.
就控制该视频的延迟时间和预览质量而言，您需要在这里进行认真的权衡.我建议使用一种混合方法，即在客户端进行编辑，但最终的弹跳/合成/无论什么操作都在服务器端进行.
There is a serious tradeoff you're making here, as far as latency to controlling that video, and quality of preview. I'd suggest a hybrid approach where editing is done client-side, but your final bounce/compositing/whatever is done server-side.
这与桌面视频编辑软件的工作方式完全不同.
This isn't unlike how desktop video editing software works anyway.
这是我现在停留的部分.我知道ffmpeg可以与nodeJS有多种不同的用法，但是我还没有找到在特定时间戳下使用ffmpeg实时播放.mp4 webm视频并发送流式视频的方法(同样，特定时间戳记).
This is the part I am stuck at now. I'm aware that ffmpeg can be used in many different ways from nodeJS, but I have not found a way to play a .mp4 webm video in realtime with ffmpeg, at a specific timestamp, and send the streaming video (again, at a certain timestamp) to the client.
是MP4还是WebM?这是两种不同的容器格式.WebM易于流式传输，因为它是直接从FFmpeg中传递出来的.MP4需要使用MOOV原子( -movflags faststart )进行监视，并且可能会有些麻烦.
Is it MP4, or is it WebM? Those are two distinct container formats. WebM is easily streamable, as piped directly out of FFmpeg. MP4 requires futzing with the MOOV atom (-movflags faststart), and can be a bit of a hassle.
无论如何，听起来您只需要在输入上设置时间戳记即可.
In any case, sounds like you just need to set timestamps on the input:
ffmpeg -ss 00:01:23 -i video.mp4 -to 00:04:56 -f webm -
我已经从ffmpeg看到了pipe:1属性，但是我找不到任何教程来使它与mp4 webm视频一起使用，以及以某种方式使用nodejs解析stdout数据并将其发送到客户端./p>
I've seen the pipe:1 attribute from ffmpeg, but I couldn't find any tutorials to get it working with an mp4 webm video, and to parse the stdout data somehow with nodejs and send it to the client.
只需使用连字符-作为输出文件名，FFmpeg将输出到STDOUT.然后，您无需在Node.js应用程序中做任何其他事情...直接输出到客户端的管道.未经测试，但您正在使用典型的Express应用程序来寻找类似的东西:
Just use a hyphen - as the output filename and FFmpeg will output to STDOUT. Then, there's nothing else you need to do in your Node.js application... pipe that output directly to the client. Untested, but you're looking for something like this, assuming a typical Express app:
app.get('/stream', (req, res, next) => {
const ffmpeg = child_process.spawn('ffmpeg', [
'-i', 'video.mp4',
'-f', 'webm',
'-'
]);
res.set('Content-Type', 'video/webm'); // TODO: Might want to set your codecs here also
ffmpeg.stdout.pipe(res);
});
即使我能使这部分工作，我仍然不知道要在某个时间戳实时播放视频.
And even if I could get that part to work, I still have no idea to play the video, in realtime, at a certain timestamp.
好吧，为此，您只是在播放一个流，所以您可以这样做:
Well, for this, you're just playing a stream so you can just do:
<video src="https://your-nodejs-server.example.com/stream" preload="none" />
preload ="none" 部分很重要，可以使其保持实时"状态.
The preload="none" part is important, to keep it "live".
所有替代方法是设置 GStreamer 管道，并可能利用其内置的在WebRTC堆栈中.这并非微不足道，但具有以下优点:潜在地降低了等待时间，并自动处理了从服务器捕捉"到实时视频的过程.如果您使用普通的视频标签，则必须通过监视缓冲的数据并管理播放速度来自己处理.
An alternative to all of this is to set up a GStreamer pipeline, and probably utilize its built-in WebRTC stack. This is not trivial, but has the advantage of potentially lower latency, and automatic handling of "catching up" to live video from the server. If you use the normal video tag, you'll have to handle that yourself by monitoring the buffered data and managing the playback speed.
我也看过ffplay ...
I've also seen ffplay...
FFplay与您的项目无关.
FFplay isn't relevant to your project.
希望这堆笔记能给您一些思考和研究的东西.
Hopefully this pile of notes will give you some things to consider and look at.
这篇关于nodejs ffmpeg在特定时间播放视频并将其流式传输到客户端的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
