OpenCV 实时流视频捕获速度很慢.如何丢帧或实时同步? [英] OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?

问题描述

我想设置一个 opencv 系统来处理 HLS 流或 RMTP 流，但是，我遇到了一个关于降低的帧速率和累积延迟的奇怪问题.就好像视频离它应该在流中的位置越来越远.

I'd like to set up an opencv system to process either HLS streams or RMTP streams, however, I am running into a strange issue regarding a reduced frame-rate and an accumulating lag. It's as if the video gets further and further behind from where it is supposed to be in the stream.

我正在寻找一种方法来与实时源保持同步，即使这意味着丢帧.

I'm looking for a way to keep up to date with a live source even if it means dropping frames.

import cv2

cap = cv2.VideoCapture()
cap.open('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8')

while (True):
    _, frame = cap.read()
    cv2.imshow("camCapture", frame)
    cv2.waitKey(1)

我已经在 VLC 上验证了流的质量，它似乎在那里工作正常.

I've validated the quality of the stream on VLC and it seems to work fine there.

.

• 我在这里做错了什么?
• 为什么这么慢?
• 如何将其同步到实时速度?

推荐答案

我的假设是抖动最有可能是由于网络限制造成的，并且发生在帧数据包丢失时.当一帧被丢弃时，这会导致程序显示最后一个好"帧，从而导致显示冻结.这可能是硬件或带宽问题，但我们可以通过软件来缓解其中的一些问题.以下是一些可能的更改:

My hypothesis is that the jitter is most likely due to network limitations and occurs when a frame packet is dropped. When a frame is dropped, this causes the program to display the last "good" frame which results in the display freezing. This is probably a hardware or bandwidth issue but we can alleviate some of this with software. Here are some possible changes:

1.设置最大缓冲区大小

我们使用 cv2.CAP_PROP_BUFFERSIZE 参数将 cv2.videoCapture() 对象设置为具有有限的缓冲区大小.这个想法是通过限制缓冲区，我们将始终拥有最新的帧.这也有助于缓解帧随机向前跳跃的问题.

We set the cv2.videoCapture() object to have a limited buffer size with the cv2.CAP_PROP_BUFFERSIZE parameter. The idea is that by limiting the buffer, we will always have the latest frame. This can also help to alleviate the problem of frames randomly jumping ahead.

2.设置帧检索延迟

目前，我认为 read() 读取速度太快，即使它在自己的专用线程中.这可能是所有帧似乎汇集在一起​​并在下一帧中突然爆发的一个原因.例如，假设在一秒的时间间隔内，它可能会产生 15 个新帧，但在接下来的一秒间隔内，仅返回 3 个帧.这可能是由于网络数据包丢帧，所以为了确保我们获得恒定的帧率，我们简单地在帧检索线程中添加了一个延迟.获得大约 ~30 FPS 的延迟可以很好地标准化"帧速率并平滑帧之间的过渡，以防出现数据包丢失.

Currently, I believe the read() is reading too fast even though it is in its own dedicated thread. This may be one reason why all the frames appear to pool up and suddenly burst in the next frame. For instance, say in a one second time interval, it may produce 15 new frames but in the next one second interval, only 3 frames are returned. This may be due to the network packet frame loss so to ensure that we obtain constant frame rates, we simply add a delay in the frame retrieval thread. A delay to obtain roughly ~30 FPS does a good job to "normalize" the frame rate and smooth the transition between frames incase there is packet loss.

注意:我们应该尝试匹配流的帧速率，但我不确定网络摄像头的 FPS 是多少，所以我只是猜测 30 FPS.此外，通常有一个直接"流链接，而不是通过中间网络服务器，这可以大大提高性能.

Note: We should try to match the frame rate of the stream but I'm not sure what the FPS of the webcam is so I just guessed 30 FPS. Also, there is usually a "direct" stream link instead of going through a intermediate webserver which can greatly improve performance.

如果您尝试使用保存的 .mp4 视频文件，您会注意到没有抖动.这证实了我的怀疑，问题很可能是由于网络延迟造成的.

If you try using a saved .mp4 video file, you will notice that there is no jitter. This confirms my suspicion that the problem is most likely due to network latency.

from threading import Thread
import cv2, time

class ThreadedCamera(object):
    def __init__(self, src=0):
        self.capture = cv2.VideoCapture(src)
        self.capture.set(cv2.CAP_PROP_BUFFERSIZE, 2)

        # FPS = 1/X
        # X = desired FPS
        self.FPS = 1/30
        self.FPS_MS = int(self.FPS * 1000)

        # Start frame retrieval thread
        self.thread = Thread(target=self.update, args=())
        self.thread.daemon = True
        self.thread.start()

    def update(self):
        while True:
            if self.capture.isOpened():
                (self.status, self.frame) = self.capture.read()
            time.sleep(self.FPS)

    def show_frame(self):
        cv2.imshow('frame', self.frame)
        cv2.waitKey(self.FPS_MS)

if __name__ == '__main__':
    src = 'https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8'
    threaded_camera = ThreadedCamera(src)
    while True:
        try:
            threaded_camera.show_frame()
        except AttributeError:
            pass

这篇关于OpenCV 实时流视频捕获速度很慢.如何丢帧或实时同步?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！