OpenCV实时流视频捕获速度很慢.如何丢帧或与实时同步? [英] OpenCV real time streaming video capture is slow. How to drop frames or get synced with real time?

问题描述

我想设置一个opencv系统来处理HLS流或RMTP流，但是，我遇到了一个奇怪的问题，即降低的帧速率和累积的滞后.好像视频从流中应该到的位置越来越远.

I'd like to set up an opencv system to process either HLS streams or RMTP streams, however, I am running into a strange issue regarding a reduced frame-rate and an accumulating lag. It's as if the video gets further and further behind from where it is supposed to be in the stream.

我正在寻找一种与实时来源保持同步的方法，即使这意味着丢帧也是如此.

I'm looking for a way to keep up to date with a live source even if it means dropping frames.

import cv2

cap = cv2.VideoCapture()
cap.open('https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8')

while (True):
    _, frame = cap.read()
    cv2.imshow("camCapture", frame)
    cv2.waitKey(1)

我已经在VLC上验证了流的质量，并且在此看来效果很好.

I've validated the quality of the stream on VLC and it seems to work fine there.

.

• 我在这里做错了什么?
• 为什么这么慢?
• 如何将其同步到实时速度?

推荐答案

我的假设是，抖动最有可能是由于网络限制所致，并在丢弃帧数据包时发生.丢帧时，程序将显示最后一个好"帧，从而导致显示冻结.这可能是硬件或带宽问题，但我们可以通过软件缓解某些问题.以下是一些可能的更改:

My hypothesis is that the jitter is most likely due to network limitations and occurs when a frame packet is dropped. When a frame is dropped, this causes the program to display the last "good" frame which results in the display freezing. This is probably a hardware or bandwidth issue but we can alleviate some of this with software. Here are some possible changes:

1.设置最大缓冲区大小

我们使用cv2.CAP_PROP_BUFFERSIZE参数将cv2.videoCapture()对象设置为具有有限的缓冲区大小.这个想法是通过限制缓冲区，我们将始终拥有最新的帧.这也有助于缓解帧随机跳到前面的问题.

We set the cv2.videoCapture() object to have a limited buffer size with the cv2.CAP_PROP_BUFFERSIZE parameter. The idea is that by limiting the buffer, we will always have the latest frame. This can also help to alleviate the problem of frames randomly jumping ahead.

2.设置帧检索延迟

当前，我相信read()的读取速度过快，即使它在其自己的专用线程中也是如此.这可能是所有帧看起来都集中起来并突然在下一帧突然爆发的原因之一.例如，在一秒钟的时间间隔中，它可能会产生15个新帧，但是在接下来的一秒钟的时间间隔中，仅返回3个帧.这可能是由于网络数据包帧丢失所致，因此为了确保我们获得恒定的帧速率，我们只需在帧检索线程中添加一个延迟即可.延迟以获得大致 FPS可以很好地标准化"帧速率，并在丢包的情况下平滑帧之间的过渡.

Currently, I believe the read() is reading too fast even though it is in its own dedicated thread. This may be one reason why all the frames appear to pool up and suddenly burst in the next frame. For instance, say in a one second time interval, it may produce 15 new frames but in the next one second interval, only 3 frames are returned. This may be due to the network packet frame loss so to ensure that we obtain constant frame rates, we simply add a delay in the frame retrieval thread. A delay to obtain roughly ~30 FPS does a good job to "normalize" the frame rate and smooth the transition between frames incase there is packet loss.

注意:我们应该尝试匹配流的帧速率，但是我不确定网络摄像头的FPS是多少，所以我只是猜到了30 FPS.另外，通常有一个直接"流链接而不是通过中间的Web服务器，这可以大大提高性能.

Note: We should try to match the frame rate of the stream but I'm not sure what the FPS of the webcam is so I just guessed 30 FPS. Also, there is usually a "direct" stream link instead of going through a intermediate webserver which can greatly improve performance.

如果尝试使用已保存的.mp4视频文件，则会发现没有抖动.这证实了我的怀疑，该问题很可能是由于网络延迟造成的.

If you try using a saved .mp4 video file, you will notice that there is no jitter. This confirms my suspicion that the problem is most likely due to network latency.

from threading import Thread
import cv2, time

class ThreadedCamera(object):
    def __init__(self, src=0):
        self.capture = cv2.VideoCapture(src)
        self.capture.set(cv2.CAP_PROP_BUFFERSIZE, 2)

        # FPS = 1/X
        # X = desired FPS
        self.FPS = 1/30
        self.FPS_MS = int(self.FPS * 1000)

        # Start frame retrieval thread
        self.thread = Thread(target=self.update, args=())
        self.thread.daemon = True
        self.thread.start()

    def update(self):
        while True:
            if self.capture.isOpened():
                (self.status, self.frame) = self.capture.read()
            time.sleep(self.FPS)

    def show_frame(self):
        cv2.imshow('frame', self.frame)
        cv2.waitKey(self.FPS_MS)

if __name__ == '__main__':
    src = 'https://videos3.earthcam.com/fecnetwork/9974.flv/chunklist_w1421640637.m3u8'
    threaded_camera = ThreadedCamera(src)
    while True:
        try:
            threaded_camera.show_frame()
        except AttributeError:
            pass

这篇关于OpenCV实时流视频捕获速度很慢.如何丢帧或与实时同步?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！