如何写opencv mat到gstreamer管道？ [英] How to write opencv mat to gstreamer pipeline?
问题描述
我想向gstreamer管道添加一些opencv进程，然后通过udpsink发送。
我可以从gstreamer读取帧： p>
//可以在管道后面添加一些插件
cv :: VideoCapture cap（v4l2src！video / x-raw ，framerate = 30/1，width = 640，height = 480，format = RGB！videoconvert！appsink）;
cv :: Mat frame;
while（ture）{
cap>>帧;
//对框架进行一些处理
}
t计算出是如何将处理后的帧传递到以下管道： appsrc！ x264enc！ mpegtsmux！ udpsink host = localhost port = 5000
我尝试了
cv :: VideoWriter writer = cv :: VideoWriter（appsrc！x264enc！mpegtsmux！udpsink host = localhost port = 5000，0，（double）30，cv :: Size（640，480） ，true）;
writer<< processedFrame;
但是，接收器端什么都不接收。 （我使用管道 $ gst-launch-1.0 udpsrc port = 5000！tsparse！tsdemux！h264parse！avdec_h264！videoconvert！ximagesink sync = false 作为接收者）
问题，有什么办法我可以调试VideoWriter？例如检查框架是否真的写入。
请注意，我在Ubuntu 14.04上使用opencv 2.4.12和gstreamer 1.2。
任何帮助都很好！
编辑：
要提供更多信息，代码，它给了 GStreamer插件：嵌入式视频播放停止;模块appsrc0报告：内部数据流错误。
#include< stdio.h&
#include< opencv2 / highgui / highgui.hpp>
#include< opencv2 / opencv.hpp>
int main（int argc，char * argv []）{
cv :: VideoCapture cap（v4l2src！video / x-raw，framerate = 30/1，width = height = 480，format = RGB！videoconvert！appsink）;
if（！cap.isOpened（））{
printf（= ERR =无法创建capture\\\
）;
return -1;
}
cv :: VideoWriter writer;
//问题在这里
writer.open（appsrc！video / x-raw，framerate = 30/1，width = 640，height = 480，format = RGB！autovideoconvert！ximagesink sync = false ，0，（double）30，cv :: Size（640，480），true）;
if（！writer.isOpened（））{
printf（= ERR =不能创建writer\\\
）;
return -1;
}
cv :: Mat frame;
int key;
while（true）{
cap>>帧;
if（frame.empty（））{
printf（no frame\\\
）;
break;
}
writer<帧;
key = cv :: waitKey（30）;
}
cv :: destroyWindow（video）;
}
显然appsrc管道有问题，但我不知道错误，因为管道 gst-launch-1.0 v4l2src！ video / x-raw，framerate = 30/1，width = 640，height = 480，format = RGB！视频转换！ ximagesink sync = false 工作正常。
回答。
关键是在 appsrc 后使用 videoconvert ，无需设置上限。因此，作者管道看起来像 appsrc！ videoconvert！ x264enc！ mpegtsmux！ udpsink host = localhost port = 5000 。
以下是从gstreamer管道读取图像，执行一些opencv映像处理和写入它回到管道。
使用此方法，您可以轻松地将任何opencv进程添加到gstreamer管道。
code> //编译：$ g ++ opencv_gst.cpp -o opencv_gst`pkg-config --cflags --libs opencv`
#include< stdio.h>
#include< opencv2 / highgui / highgui.hpp>
#include< opencv2 / opencv.hpp>
int main（int argc，char ** argv）{
//原始gstreamer管道：
// == Sender ==
/ / gst-launch-1.0 v4l2src
//！ video / x-raw，framerate = 30/1，width = 640，height = 480，format = RGB
//！ videoconvert
//！ x264enc noise-reduction = 10000 tune = zerolatency byte-stream = true threads = 4
//！ mpegtsmux
//！ udpsink host = localhost port = 5000
//
// == Receiver ==
// gst-launch-1.0 -ve udpsrc port = 5000
//！ tsparse！ tsdemux
//！ h264parse！ avdec_h264
//！ videoconvert
//！ ximagesink sync = false
//发件人管道的第一部分
cv :: VideoCapture cap（v4l2src！video / x-raw，framerate = 30/1，width = 640，height = 480，format = RGB！videoconvert！appsink）;
if（！cap.isOpened（））{
printf（= ERR =无法创建视频捕获\\\
）;
return -1;
}
//发送者管道的第二部分
cv :: VideoWriter writer;
writer.open（appsrc！videoconvert！x264enc noise-reduction = 10000 tune = zerolatency byte-stream = true threads = 4！mpegtsmux！udpsink host = localhost port = 9999
，0， ）30，cv :: Size（640，480），true）;
if（！writer.isOpened（））{
printf（= ERR =无法创建视频编写器\\\
）;
return -1;
}
cv :: Mat frame;
int key;
while（true）{
cap>>帧;
if（frame.empty（））
break;
/ *在此处理帧* /
writer<帧;
key = cv :: waitKey（30）;
}
}
希望这有帮助。 ;）
I want to add some opencv processes to a gstreamer pipeline and then send it over udpsink.
I'm able to read frames from gstreamer like this:
// may add some plugins to the pipeline later
cv::VideoCapture cap("v4l2src ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! videoconvert ! appsink");
cv::Mat frame;
while(ture){
cap >> frame;
// do some processing to the frame
}
But what can't figure out is how to pass the processed frame to the following pipeline: appsrc ! x264enc ! mpegtsmux ! udpsink host=localhost port=5000
I've tried
cv::VideoWriter writer = cv::VideoWriter("appsrc ! x264enc ! mpegtsmux ! udpsink host=localhost port=5000", 0, (double)30, cv::Size(640, 480), true);
writer << processedFrame;
However, the receiver side receives nothing. (I uses the pipeline $gst-launch-1.0 udpsrc port=5000 ! tsparse ! tsdemux ! h264parse ! avdec_h264 ! videoconvert ! ximagesink sync=false as receiver)
My question is, can I pass processed opencv Mat to a gstreamer pipeline and let it do some encoding, and then send over network through udpsink? If yes, how do I achieve this?
Side question, is there any way I can debug a VideoWriter? Such as checking if frames are actually written into it.
Note that I'm using opencv 2.4.12 and gstreamer 1.2 on ubuntu 14.04.
Any help are great!
EDIT:
To provide more info, I tested the following code, and it gave GStreamer Plugin: Embedded video playback halted; module appsrc0 reported: Internal data flow error.
#include <stdio.h>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/opencv.hpp>
int main(int argc, char *argv[]){
cv::VideoCapture cap("v4l2src ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! videoconvert ! appsink");
if (!cap.isOpened()) {
printf("=ERR= can't create capture\n");
return -1;
}
cv::VideoWriter writer;
// problem here
writer.open("appsrc ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! autovideoconvert ! ximagesink sync=false", 0, (double)30, cv::Size(640, 480), true);
if (!writer.isOpened()) {
printf("=ERR= can't create writer\n");
return -1;
}
cv::Mat frame;
int key;
while (true) {
cap >> frame;
if (frame.empty()) {
printf("no frame\n");
break;
}
writer << frame;
key = cv::waitKey( 30 );
}
cv::destroyWindow( "video" );
}
Apparently there's something wrong with the appsrc pipeline, but I have no idea what went wrong because the pipeline gst-launch-1.0 v4l2src ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! videoconvert ! ximagesink sync=false works fine.
After hours of searching and testing, I finally got the answer.
The key is to use only videoconvert after appsrc, no need to set caps. Therefore, a writer pipeline would look like appsrc ! videoconvert ! x264enc ! mpegtsmux ! udpsink host=localhost port=5000.
Following is a sample code that reads images from a gstreamer pipeline, doing some opencv image processing and write it back to the pipeline.
With this method, you can add any opencv process to a gstreamer pipeline easily.
// Compile with: $ g++ opencv_gst.cpp -o opencv_gst `pkg-config --cflags --libs opencv`
#include <stdio.h>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/opencv.hpp>
int main(int argc, char** argv) {
// Original gstreamer pipeline:
// == Sender ==
// gst-launch-1.0 v4l2src
// ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB
// ! videoconvert
// ! x264enc noise-reduction=10000 tune=zerolatency byte-stream=true threads=4
// ! mpegtsmux
// ! udpsink host=localhost port=5000
//
// == Receiver ==
// gst-launch-1.0 -ve udpsrc port=5000
// ! tsparse ! tsdemux
// ! h264parse ! avdec_h264
// ! videoconvert
// ! ximagesink sync=false
// first part of sender pipeline
cv::VideoCapture cap("v4l2src ! video/x-raw, framerate=30/1, width=640, height=480, format=RGB ! videoconvert ! appsink");
if (!cap.isOpened()) {
printf("=ERR= can't create video capture\n");
return -1;
}
// second part of sender pipeline
cv::VideoWriter writer;
writer.open("appsrc ! videoconvert ! x264enc noise-reduction=10000 tune=zerolatency byte-stream=true threads=4 ! mpegtsmux ! udpsink host=localhost port=9999"
, 0, (double)30, cv::Size(640, 480), true);
if (!writer.isOpened()) {
printf("=ERR= can't create video writer\n");
return -1;
}
cv::Mat frame;
int key;
while (true) {
cap >> frame;
if (frame.empty())
break;
/* Process the frame here */
writer << frame;
key = cv::waitKey( 30 );
}
}
Hope this helps. ;)
这篇关于如何写opencv mat到gstreamer管道？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
