pyav/libav/ffmpeg 当来自实时源的帧处理速度不够快时会发生什么 [英] pyav / libav / ffmpeg what happens when frame from live source are not processed fast enough

问题描述

我正在使用 pyav 来处理实时 RTSP 流:

I am using pyav to process a live RTSP stream:

import av
import time

URL = "RTSP_url"
container = av.open(
            url, 'r',
            options={
                'rtsp_transport': 'tcp',
                'stimeout': '5000000',
                'max_delay': '5000000',
            }
        )

for packet in self.container.demux(video=0):
    for frame in packet.decode():
        # do something
        time.sleep(10)

如果我做某事太慢会怎样?帧/数据包是否被丢弃或被缓冲?

What happens if I do something too slow? Are frames / packets dropped or are they buffered?

我想同样的问题也适用于 libav 或 ffmpeg.

I guess the same question would apply to libav or ffmpeg.

推荐答案

tcp 是一种具有内置流量控制的有保证的交付协议.如果您没有像接收到的那样快处理传入的数据，tcp 堆栈将缓冲数据，直到其缓冲区已满，此时 tcp 协议将让发送方知道它不能再接收任何数据.如果这种情况继续下去，发送方的输出缓冲区最终会填满，然后由发送方决定要做什么.

tcp is a guaranteed delivery protocol with built-in flow control. If you do not process the incoming data as fast as it is received, the tcp stack will buffer the data until its buffers are full at which time the tcp protocol will let the sender know that it cannot receive any more data. If this continues, the sender's output buffers will eventually fill up and then it is up to the sender to decide what to do.

此时的 IP 摄像机可能会丢帧，甚至可能会断开连接.大多数 IP 摄像机还使用常态通过 RTSP 流发送的 RTCP 数据包的保持活动机制.相机可能会发送发送者报告，而接收者应该发回接收者报告.如果相机在超时内没有收到接收器报告，它将断开连接.我必须假设 av 库或 ffmpeg 正在这样做.

An IP camera at that point may throw frames away or it may even drop the connection. Most IP cameras also use a keep-alive mechanism typically via RTCP packets sent over the RTSP stream. The camera may send Sender Reports and the receiver should send back Receiver Reports. If the camera does not get a Receiver Report within a timeout, it will drop the connection. I would have to assume that either the av library or ffmpeg is doing that.

你可能不想做time.sleep(10).

如果你真的觉得你需要丢弃数据包，那么你可以在调用 decode 之前检查你的数据包，看看你是否落后.如果您落后太多，您可以丢弃不是关键帧的数据包，直到您赶上.效果是视频会有跳跃.

If you really feel that you need to discard packets, then you could examine your packets before calling decode to see if you are falling behind. If you are getting too far behind, you can discard packets that are not key frames until you catch up. The effect will be that the video will have jumps in it.

这篇关于pyav/libav/ffmpeg 当来自实时源的帧处理速度不够快时会发生什么的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！