如何有效地在Qt中显示OpenCV视频？ [英] How to efficiently display OpenCV video in Qt?
问题描述
我在OpenCV的帮助下捕获来自ip摄像机的多个流。当我尝试显示这些流从一个OpenCV窗口（ cv :: namedWindow（...）），它的工作没有任何问题（我已经尝试最多4个流far）。
I'm capturing multiple streams from ip cameras with the help of OpenCV. When i try to display these stream from an OpenCV window(cv::namedWindow(...)), it works without any problem (i have tried up to 4 streams so far).
当我试图在Qt小部件中显示这些流问题出现。因为捕获是在另一个线程中完成的，我必须使用信号槽机制，以更新QWidget（在主线程）。
The problem arises when i try to show these streams inside a Qt widget. Since the capturing is done in another thread, i have to use the signal slot mechanism in order to update the QWidget(which is in main thread).
基本上，i emit来自捕获线程的新捕获的帧和GUI线程中的槽捕获它。当我打开4个视频流时，我无法像以前一样顺利地显示视频。
Basically, i emit the newly captured frame from the capture thread and a slot in the GUI thread catches it. When i open 4 streams, i can not display the videos smoothly like before.
这里是发射器：
void capture::start_process() {
m_enable = true;
cv::Mat frame;
while(m_enable) {
if (!m_video_handle->read(frame)) {
break;
}
cv::cvtColor(frame, frame,CV_BGR2RGB);
qDebug() << "FRAME : " << frame.data;
emit image_ready(QImage(frame.data, frame.cols, frame.rows, frame.step, QImage::Format_RGB888));
cv::waitKey(30);
}
}
这是我的位置：
void widget::set_image(QImage image) {
img = image;
qDebug() << "PARAMETER IMAGE: " << image.scanLine(0);
qDebug() << "MEMBER IMAGE: " << img.scanLine(0);
}
问题似乎是连续复制QImages的开销。虽然QImage使用隐式共享，当我通过 qDebug（）消息比较图像的数据指针，我看到不同的地址。
The problem seems like the overhead of copying QImages continuously. Although QImage uses implicit sharing, when i compare the data pointers of images via qDebug() messages, i see different addresses.
1-有什么办法直接嵌入OpenCV窗口到QWidget吗？
1- Is there any way to embed OpenCV window directly into QWidget ?
2-处理显示多个视频的最有效的方法是什么？例如，视频管理系统如何在同一时间显示多达32台摄像机？
2- What is the most efficient way to handle displaying multiple videos? For example, how video management systems show up to 32 cameras in the same time ?
3-必须走什么路？
推荐答案
使用 QImage :: scanLine 强制执行深拷贝，因此至少应使用 constScanLine ，或者更好的是，将插槽的签名更改为：
Using QImage::scanLine forces a deep copy, so at the minimum, you should use constScanLine, or, better yet, change the slot's signature to:
void widget::set_image(const QImage & image);
当然，你的问题会变成别的：QImage实例指向一个框架的数据
Of course, your problem then becomes something else: the QImage instance points to the data of a frame that lives in another thread, and can (and will) change at any moment.
有一个解决方案：需要使用在堆上分配的新鲜帧，并且可以在任何时候改变。该框架需要在 QImage 中捕获。 QScopedPointer 用于防止内存泄漏，直到 QImage 拥有框架的所有权。
There is a solution for that: one needs to use fresh frames allocated on the heap, and the frame needs to be captured within QImage. QScopedPointer is used to prevent memory leaks until the QImage takes ownership of the frame.
static void matDeleter(void* mat) { delete static_cast<cv::Mat*>(mat); }
class capture {
Q_OBJECT
bool m_enable;
...
public:
Q_SIGNAL void image_ready(const QImage &);
...
};
void capture::start_process() {
m_enable = true;
while(m_enable) {
QScopedPointer<cv::Mat> frame(new cv::Mat);
if (!m_video_handle->read(*frame)) {
break;
}
cv::cvtColor(*frame, *frame, CV_BGR2RGB);
// Here the image instance takes ownership of the frame.
const QImage image(frame->data, frame->cols, frame->rows, frame->step,
QImage::Format_RGB888, matDeleter, frame.take());
emit image_ready(image);
cv::waitKey(30);
}
}
当然，因为Qt提供了原生消息分派 QObject 捕获过程。下面是一个完整的测试示例。
Of course, since Qt provides native message dispatch and a Qt event loop by default in a QThread, it's a simple matter to use QObject for the capture process. Below is a complete, tested example.
捕获，转换和查看器都在自己的线程中运行。由于 cv :: Mat 是一个具有原子，线程安全访问的隐式共享类，因此使用它。
The capture, conversion and viewer all run in their own threads. Since cv::Mat is an implicitly shared class with atomic, thread-safe access, it's used as such.
转换器具有不处理过时帧的选项 - 如果转换仅用于显示目的，则非常有用。
The converter has an option of not processing stale frames - useful if conversion is only done for display purposes.
查看器在gui线程中运行，并正确删除过时的帧。
The viewer runs in the gui thread and correctly drops stale frames. There's never a reason for the viewer to deal with stale frames.
如果你要收集数据保存到磁盘，你应该以高优先级运行捕获线程。您还应该检查openCV apis，看看是否有一种方法可以将本机相机数据转储到磁盘。
If you were to collect data to save to disk, you should run the capture thread at high priority. You should also inspect openCV apis to see if there's a way of dumping the native camera data to disk.
为了加快转换速度，您可以使用gpu加速类openCV。
To speed up conversion, you could use the gpu-accelerated classes in openCV.
#include <QApplication>
#include <QBasicTimer>
#include <QThread>
#include <QImage>
#include <QWidget>
#include <QPainter>
#include <QPaintEvent>
#include <QDebug>
#include <opencv2/opencv.hpp>
Q_DECLARE_METATYPE(cv::Mat)
class Capture : public QObject {
Q_OBJECT
QBasicTimer m_timer;
QScopedPointer<cv::VideoCapture> m_videoCapture;
public:
Capture(QObject * parent = 0) : QObject(parent) {}
Q_SIGNAL void started();
Q_SLOT void start(int cam = 0) {
if (!m_videoCapture)
m_videoCapture.reset(new cv::VideoCapture(cam));
if (m_videoCapture->isOpened()) {
m_timer.start(0, this);
emit started();
}
}
Q_SLOT void stop() { m_timer.stop(); }
Q_SIGNAL void matReady(const cv::Mat &);
private:
void timerEvent(QTimerEvent * ev) {
if (ev->timerId() != m_timer.timerId()) return;
cv::Mat frame;
if (!m_videoCapture->read(frame)) { // Blocks until a new frame is ready
m_timer.stop();
return;
}
emit matReady(frame);
}
};
class Converter : public QObject {
Q_OBJECT
QBasicTimer m_timer;
cv::Mat m_frame;
bool m_processAll;
static void matDeleter(void* mat) { delete static_cast<cv::Mat*>(mat); }
void queue(const cv::Mat & frame) {
if (!m_frame.empty()) qDebug() << "Converter dropped frame!";
m_frame = frame;
if (! m_timer.isActive()) m_timer.start(0, this);
}
void process(cv::Mat frame) {
cv::resize(frame, frame, cv::Size(), 0.3, 0.3, cv::INTER_AREA);
cv::cvtColor(frame, frame, CV_BGR2RGB);
const QImage image(frame.data, frame.cols, frame.rows, frame.step,
QImage::Format_RGB888, &matDeleter, new cv::Mat(frame));
Q_ASSERT(image.constBits() == frame.data);
emit imageReady(image);
}
void timerEvent(QTimerEvent * ev) {
if (ev->timerId() != m_timer.timerId()) return;
process(m_frame);
m_frame.release();
m_timer.stop();
}
public:
explicit Converter(QObject * parent = 0) : QObject(parent), m_processAll(true) {}
void setProcessAll(bool all) { m_processAll = all; }
Q_SIGNAL void imageReady(const QImage &);
Q_SLOT void processFrame(const cv::Mat & frame) {
if (m_processAll) process(frame); else queue(frame);
}
};
class ImageViewer : public QWidget {
Q_OBJECT
QImage m_img;
void paintEvent(QPaintEvent *) {
QPainter p(this);
p.drawImage(0, 0, m_img);
m_img = QImage();
}
public:
ImageViewer(QWidget * parent = 0) : QWidget(parent) {
setAttribute(Qt::WA_OpaquePaintEvent);
}
Q_SLOT void setImage(const QImage & img) {
if (!m_img.isNull()) qDebug() << "Viewer dropped frame!";
m_img = img;
if (m_img.size() != size()) setFixedSize(m_img.size());
update();
}
};
int main(int argc, char *argv[])
{
qRegisterMetaType<cv::Mat>();
QApplication app(argc, argv);
ImageViewer view;
Capture capture;
Converter converter;
QThread captureThread, converterThread;
// Everything runs at the same priority as the gui, so it won't supply useless frames.
converter.setProcessAll(false);
captureThread.start();
converterThread.start();
capture.moveToThread(&captureThread);
converter.moveToThread(&converterThread);
converter.connect(&capture, SIGNAL(matReady(cv::Mat)), SLOT(processFrame(cv::Mat)));
view.connect(&converter, SIGNAL(imageReady(QImage)), SLOT(setImage(QImage)));
view.show();
QObject::connect(&capture, &Capture::started, [](){ qDebug() << "capture started"; });
QMetaObject::invokeMethod(&capture, "start");
int rc = app.exec();
captureThread.quit();
converterThread.quit();
captureThread.wait();
converterThread.wait();
return rc;
}
#include "main.moc"
这篇关于如何有效地在Qt中显示OpenCV视频？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
