如何使用OpenCV解决图像处理相机的IO延迟 [英] How to solve image processing camera IO delay with OpenCV

问题描述

我有一个像这样的OpenCV程序:

I am having an OpenCV program which works like this:

VideoCapture cap(0);
Mat frame;
while(true) {
  cap >> frame;
  myprocess(frame);
}

问题是如果myprocess花费的时间长于摄像机的IO间隔，则捕获的帧将被延迟，无法使帧与实时同步.

The problem is if myprocess takes a long time which longer than camera's IO interval, the captured frame be delayed, cannot get the frame synchronized with the real time.

因此，我认为要解决此问题，应使摄像机流和myprocess并行运行.一个线程进行IO操作，另一个线程进行CPU计算.相机完成捕获后，发送到工作线程进行处理.

So, I think to solve this problem, should make the camera streaming and myprocess run parallelly. One thread does IO operation, another does CPU computing. When the camera finished capture, send to work thread to processing.

这个主意对吗?解决这个问题有更好的策略吗?

Is this idea right? Any better strategy to solve this problem?

演示:

int main(int argc, char *argv[])
{
    cv::Mat buffer;
    cv::VideoCapture cap;
    std::mutex mutex;
    cap.open(0);
    std::thread product([](cv::Mat& buffer, cv::VideoCapture cap, std::mutex& mutex){
        while (true) { // keep product the new image
            cv::Mat tmp;
            cap >> tmp;
            mutex.lock();
            buffer = tmp.clone(); // copy the value
            mutex.unlock();
        }
    }, std::ref(buffer), cap, std::ref(mutex));
    product.detach();

    while (cv::waitKey(20)) { // process in the main thread
        mutex.lock();
        cv::Mat tmp = buffer.clone(); // copy the value
        mutex.unlock();
        if(!tmp.data)
            std::cout<<"null"<<std::endl;
        else {
            std::cout<<"not null"<<std::endl;
            cv::imshow("test", tmp);
        }

    }
    return 0;
}

或者使用线程不断清除缓冲区.

Or use a thread keep clearing the buffer.

int main(int argc, char *argv[])
{
    cv::Mat buffer;
    cv::VideoCapture cap;
    std::mutex mutex;
    cap.open(0);
    std::thread product([](cv::Mat& buffer, cv::VideoCapture cap, std::mutex& mutex){
        while (true) { // keep product the new image
            cap.grab();
        }
    }, std::ref(buffer), cap, std::ref(mutex));
    product.detach();
    int i;
    while (true) { // process in the main thread
        cv::Mat tmp;
        cap.retrieve(tmp);
        if(!tmp.data)
            std::cout<<"null"<<i++<<std::endl;
        else {
            cv::imshow("test", tmp);
        }
        if(cv::waitKey(30) >= 0) break;
    }
    return 0;
}

我认为第二个演示应基于

The second demo I thought shall be work base on https://docs.opencv.org/3.0-beta/modules/videoio/doc/reading_and_writing_video.html#videocapture-grab, but it's not...

推荐答案

在具有多目标跟踪的项目中，我使用了2个帧缓冲(cv :: Mat frame [2])和2个线程:

In project with Multitarget tracking I used 2 buffers for frame (cv::Mat frames[2]) and 2 threads:

1. 一个用于捕获下一帧并检测对象的线程.

1. One thread for capturing the next frame and detect objects.

第二个线程，用于跟踪检测到的对象并在框架上绘制结果.

Second thread for tracking the detected objects and draw result on frame.

我使用index = [0,1]进行缓冲区交换，并且此索引受互斥锁保护.为了通知工作结束，使用了两个条件变量.

I used index = [0,1] for the buffers swap and this index was protected with mutex. For signalling about the end of work was used 2 condition variables.

CatureAndDetect首先使用frame [capture_ind]缓冲区，而Tracking使用先前的frame [1-capture_ind]缓冲区.下一步-切换缓冲区:capture_ind = 1-capture_ind.

First works CatureAndDetect with frames[capture_ind] buffer and Tracking works with previous frames[1-capture_ind] buffer. Next step - switch the buffers: capture_ind = 1 - capture_ind.

您可以在这里进行此项目吗:多目标-跟踪器.

Do you can this project here: Multitarget-tracker.

这篇关于如何使用OpenCV解决图像处理相机的IO延迟的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！