Ffmpeg-如何强制整个帧输出MJPEG? [英] Ffmpeg - How to force MJPEG output of whole frames?
问题描述
我正在与ffmpeg一起处理来自远程摄像机的传入MPEGTS流，并使用我的应用程序将其传递给多个客户端.
I'm working with ffmpeg to process an incoming MPEGTS stream from remote cameras, and deliver it to multiple clients using my app.
从技术上讲，我使用ffmpeg将传入流转换为MJPEG输出，并将数据块(来自ffmpeg进程stdout)通过管道传输到客户端http响应上的可写流.
Technically, I'm using ffmpeg to convert the incoming stream to an MJPEG output, and piping the data chunks (from the ffmpeg process stdout) to a writeable stream on the client http response.
但是，我面临一个问题-并非所有数据块都代表完整的整个"框架.因此，将它们在浏览器中连续显示会导致视频随机闪烁，且帧数为半完整.我知道这一点是因为打印每个块的长度时，大多数时候结果都是一个大值(X)，但是我时不时地得到2个连续的块，其长度为(2/5X)，然后是(3/5X).
However, I'm facing a problem- not all data chunks represent a full 'whole' frame. thus, displaying them in a row in the browser, results in a flickering video, with half-complete frames, on a random basis. I know this because when printing each chunk length, results most of the time in a big value (X), but every now and then I get 2 consecutive chunks with length (2/5X) followed by (3/5X) for example.
所以问题-是否有一种方法可以强制ffmpeg进程仅输出整个帧?如果没有，我是否有办法手动"检查每个数据块并寻找标题/元数据/标志来指示帧的开始/结束?
So the question - is there a way to force the ffmpeg process to output only whole frames? if not, is there a way for me to check each data chunk 'manually' and look for headers/metadata/flags to indicate frame start/end?
我的用于输出MJPEG的ffmpeg命令是:
my ffmpeg command for outputting MJPEG is:
ffmpeg -i - -c:v mjpeg -f mjpeg -
解释:
-i-" :(输入)是进程的标准输入(不是静态文件)
"-i -" : (input) is the stdin of the process (and not a static file)
-c:v mjpeg":使用mjpeg编解码器
"-c:v mjpeg" : using the mjpeg codec
-f mjpeg":输出将为mjpeg格式
"-f mjpeg" : output will be in the mjpeg format
-":未指定输出(文件或url)-将为标准输出
"-" : output not specified (file or url) - will be the process stdout
修改:这是一些console.log打印以可视化问题:
here are some console.log prints to visualize the problem:
%%% FFMPEG Info %%%
frame= 832 fps= 39 q=24.8 q=29.0 size= 49399kB time=00:00:27.76 bitrate=14577.1kbits/s speed=1.29x
data.length: 60376
data.length: 60411
data.length: 60465
data.length: 32768
data.length: 27688
data.length: 32768
data.length: 27689
data.length: 60495
data.length: 60510
data.length: 60457
data.length: 59811
data.length: 59953
data.length: 59889
data.length: 59856
data.length: 59936
data.length: 60049
data.length: 60091
data.length: 60012
%%% FFMPEG Info %%%
frame= 848 fps= 38 q=24.8 q=29.0 size= 50340kB time=00:00:28.29 bitrate=14574.4kbits/s speed=1.28x
data.length: 60025
data.length: 60064
data.length: 60122
data.length: 60202
data.length: 60113
data.length: 60211
data.length: 60201
data.length: 60195
data.length: 60116
data.length: 60167
data.length: 60273
data.length: 60222
data.length: 60223
data.length: 60267
data.length: 60329
%%% FFMPEG Info %%%
frame= 863 fps= 38 q=24.8 q=29.0 size= 51221kB time=00:00:28.79 bitrate=14571.9kbits/s speed=1.27x
如您所见，整个帧大约为60k(我的指示是我正在浏览器中查看的干净视频流)，但是输出时不时地包含2个连续的块，总计约60k.交付给浏览器时，它们就是半帧".
As you can see, a whole frame is about ~60k (my indication is a clean video stream i'm viewing on the browser), but every now and then the output consists of 2 consecutive chunks that add up to ~60k. when delivered to the browser, these are 'half frames'.
推荐答案
在此处以及StackExchange上的注释之后，似乎从ffmpeg进程输出的MJPEG流应该由整个帧组成.收听ffmpeg ChildProcess stdout会产生大小不同的数据块-这意味着它们并不总是代表整个帧(完整的JPEG)图像.
Following the comments here and on StackExchange, it seems that the MJPEG stream outputted from the ffmpeg process should consist of whole frames. listening to the ffmpeg ChildProcess stdout yields data chunks of varying size- which means they dont always represent a whole frame (full JPEG) image.
因此，我不只是将它们推送给消费者(当前是一个显示视频流的Web浏览器)，而是编写了一些代码来处理内存中的半块"并将它们附加在一起，直到框架完成.
So instead of just pushing them to the consumer (currently a web browser showing the video stream) I wrote a bit of code to handle 'half-chunks' in memory and append them together until the frame is complete.
这似乎可以解决问题，因为视频中没有出现闪烁.
This seems to solve the problem, as I'm getting no flickering in the video.
const _SOI = Buffer.from([0xff, 0xd8]);
const _EOI = Buffer.from([0xff, 0xd9]);
private size: number = 0;
private chunks: any[] = [];
private jpegInst: any = null;
private pushWholeMjpegFrame(chunk: any): void {
const chunkLength = chunk.length;
let pos = 0;
while (true) {
if (this.size) {
const eoi = chunk.indexOf(_EOI);
if (eoi === -1) {
this.chunks.push(chunk);
this.size += chunkLength;
break;
} else {
pos = eoi + 2;
const sliced = chunk.slice(0, pos);
this.chunks.push(sliced);
this.size += sliced.length;
this.jpegInst = Buffer.concat(this.chunks, this.size);
this.chunks = [];
this.size = 0;
this.sendJpeg();
if (pos === chunkLength) {
break;
}
}
} else {
const soi = chunk.indexOf(_SOI, pos);
if (soi === -1) {
break;
} else {
pos = soi + 500;
}
const eoi = chunk.indexOf(_EOI, pos);
if (eoi === -1) {
const sliced = chunk.slice(soi);
this.chunks = [sliced];
this.size = sliced.length;
break;
} else {
pos = eoi + 2;
this.jpegInst = chunk.slice(soi, pos);
this.sendJpeg();
if (pos === chunkLength) {
break;
}
}
}
}
}
如果可以改善和优化我的解决方案，我将希望获得更多的教育意见，以及更多有关问题根源的知识，也许是一种使期望的行为脱颖而出的方法-装有ffmpeg的盒子因此，请随时通过提供更多答案和评论来使这个问题继续存在.
I would love to get some more educated input on my solution if it can be improved and optimized, as well as some more knowledge as to the origin of the problem and perhaps a way to get the desired behavior out-of-the-box with ffmpeg, so feel free to keep this question alive with more answers and comments.
这篇关于Ffmpeg-如何强制整个帧输出MJPEG?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
