MJPEG流和解码 [英] MJPEG streaming and decoding
问题描述
我想从IP摄像机接收JPEG图像(通过RTSP)。为此,我在OpenCV中尝试了 cvCreateFileCapture_FFMPEG
。但ffmpeg似乎有一些问题与流的MJPEG格式(因为它自动尝试检测流信息),我结束了以下错误
mjpeg:不支持的编码类型
然后,我决定使用live555流。到目前为止,我可以通过openRTSP成功建立流和捕获(非解码)图像。
问题是如何在我的应用程序中,例如在OpenCV中这样做。如何在OpenCV中使用openRTSP获取图像并将其保存为JPEG格式?
我听说openRTSP的数据可以发送到缓冲区(或命名管道),然后在OpenCV的 IplImage
。但我不知道该怎么做。
我真的很感谢任何关于这个问题的帮助/建议。我需要以下问题的答案:
- 如何禁用ffmpeg的自动流信息检测并指定自己的格式(mjpeg)或
- 如何在OpenCV中使用openRTSP?
b $ b
这是一个Axis IP摄像机吗?无论哪种方式,大多数提供 MPEG4 RTSP流的IP摄像机都可以使用OpenCV使用 cvCreateFileCapture_FFMPEG 解码。然而,ffmpeg解码器的 MJPEG 编解码器有一个广为人知的未解决的问题。我确定您会收到类似
的错误 [ingenient @ 0x97d20c0]编解码器参数(视频:mjpeg)
选项1:使用opencv,libcurl和libjpeg
要在opencv中查看mjpeg流,请查看以下实现
http://www.eecs.ucf.edu/~rpatrick/code/onelinksys.c
或
http://cse.unl.edu/~ rpatrick / code / onelinksys.c
Option2:使用gstreamer(无opencv)
如果您的目标是查看或保存jpeg图片,我建议您查看gstreamer
要查看可以执行如下的媒体管道字符串
gst-launch -v souphttpsrc location =http:// [ip]:[port ] / [dir] /xxx.cgido-timestamp = true is_live = true! multipartdemux! jpegdec! ffmpegcolorspace! autovideosink
For RTSP
gst-launch -v rtspsrc location =rtsp:// [user]:[pass] @ [ip]:[port] / [dir] /xxx.ampdebug = 1! rtpmp4vdepay! mpeg4videoparse! ffdec_mpeg4! ffmpegcolorspace! autovideosink
要使用C API,请参阅
对于一个简单的例子,看看我在rtsp上的其他帖子构造gstreamer C API媒体管道(这与gst-launch字符串相同,但实现为C API)
将保存 MJPEG流作为多个图像管道(让我们将垂直翻转< > BIN ,并将 PADS 连接到上一个和下一个 BINS 以使其更好)
gst-launch souphttpsrc location =http:// [ip]:[port] / [dir] /xxx.cgido-timestamp = true is_live = true! multipartdemux! jpegdec! videoflip方法=垂直翻转! jpegenc! multifilesink location = image-out-%05d.jpg
也许值得看看 gst-opencv :使用gstreamer,命名管道和opencv
在Linux上可以得到mjpeg流,并将其转换为mpeg4并将其提供给命名管道。然后从opencv中的命名管道读取数据
步骤1.创建命名管道
mkfifo stream_fifo
步骤2.创建opencvvideo_test.c
//使用gcc -ggdb编译pkg-config --cflags --libs opencv` opencvvideo_test.c -o opencvvideo_test
#include< stdio.h>
#includehighgui.h
#includecv.h
int main(int argc,char ** argv){
IplImage * frame;
int key;
/ *提供要播放的AVI文件* /
assert(argc == 2);
/ *加载AVI文件* /
CvCapture * capture = cvCreateFileCapture(argv [1]); // cvCaptureFromAVI(argv [1]);
/ *总是检查* /
if(!capture)return 1;
/ * get fps,需要设置延迟* /
int fps =(int)cvGetCaptureProperty(capture,CV_CAP_PROP_FPS);
int frameH =(int)cvGetCaptureProperty(capture,CV_CAP_PROP_FRAME_HEIGHT);
int frameW =(int)cvGetCaptureProperty(capture,CV_CAP_PROP_FRAME_WIDTH);
/ * display video * /
cvNamedWindow(video,CV_WINDOW_AUTOSIZE);
while(key!='q'){
double t1 =(double)cvGetTickCount();
/ * get a frame * /
frame = cvQueryFrame(capture);
double t2 =(double)cvGetTickCount();
printf(time:%gms fps:%.2g \\\
,(t2-t1)/(cvGetTickFrequency()* 1000。),1000 ./((t2_t1)/(cvGetTickFrequency * 1000。)));
/ *总是检查* /
if(!frame)break;
/ *显示帧* /
cvShowImage(video,frame);
/ *如果用户按'q'* /
key = cvWaitKey(1000 / fps)
}
/ *可用内存* /
cvReleaseCapture(& capture);
cvDestroyWindow(video);
return 0;
}
步骤3.准备使用gstreamer从MJPEG转换为MPEG4 frames critical)
gst-launch -v souphttpsrc location =http://< ip> / cgi_bin /< mjpeg> ; .cgido-timestamp = true is_live = true! multipartdemux! jpegdec!排队! videoscale! 'video / x-raw-yuv,width = 640,height = 480'!排队!视频! 'video / x-raw-yuv,framerate = 30/1'!排队! ffmpegcolorspace! 'video / x-raw-yuv,format =(fourcc)I420'! ffenc_mpeg4!排队! fileink location = stream_fifo
步骤4.在OpenCV中显示流
./opencvvideo_test stream_fifo
I want to receive JPEG images from an IP camera (over RTSP). For this, I tried cvCreateFileCapture_FFMPEG
in OpenCV. But ffmpeg seems to have some problem with the MJPEG format of the streaming (since it automatically tries to detect the streaming info) and I end up with the following error
mjpeg: unsupported coding type
I, then, decided to use live555 for streaming. Till now, I can successfully establish streaming and capture (non-decoded) images through openRTSP.
The question is how can I do this in my application, e.g., in OpenCV. How can I use openRTSP in OpenCV to get images and save them in JPEG format?
I have heard that the data from openRTSP can be sent to a buffer (or a named pipe) and then read in OpenCV's IplImage
. But I don't know how to do this.
I will really appreciate any help/suggestion in about this problem. I need answers of either of the following questions:
- How can I disable ffmpeg's automatic stream information detection and specify my own format (mjpeg), or
- How can I use openRTSP in OpenCV?
Regards,
Is this an Axis IP camera? Either way, most IP cameras that provide MPEG4 RTSP stream that can be decoded using OpenCV using cvCreateFileCapture_FFMPEG. However, ffmpeg decoder's MJPEG codec has a widely known unresolved issues. I am sure you would have received an error similar to
[ingenient @ 0x97d20c0]Could not find codec parameters (Video: mjpeg)
Option1 : Using opencv, libcurl and libjpeg
To view mjpeg stream in opencv take a look at the following implementation
http://www.eecs.ucf.edu/~rpatrick/code/onelinksys.c or http://cse.unl.edu/~rpatrick/code/onelinksys.c
Option2: Using gstreamer (no opencv)
I would recommend looking at gstreamer if your goal is to just view or save jpeg images
To view MJPEG stream one may execute media pipeline string as follows
gst-launch -v souphttpsrc location="http://[ip]:[port]/[dir]/xxx.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! ffmpegcolorspace ! autovideosink
For RTSP
gst-launch -v rtspsrc location="rtsp://[user]:[pass]@[ip]:[port]/[dir]/xxx.amp" debug=1 ! rtpmp4vdepay ! mpeg4videoparse ! ffdec_mpeg4 ! ffmpegcolorspace! autovideosink
To work with C API see
For a simple example take a look at my other post on rtsp for constructing gstreamer C API media pipeline (This is same as gst-launch string but rather implemented as a C API)
Playing RTSP with python-gstreamer
To save MJPEG stream as multiple images the pipeline (Let us put a vertical flip BIN and connect the PADS to the previous and the next BINS to make it fancier)
gst-launch souphttpsrc location="http://[ip]:[port]/[dir]/xxx.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! videoflip method=vertical-flip ! jpegenc ! multifilesink location=image-out-%05d.jpg
Also maybe worthwhile have a look at gst-opencv
UPDATE:
Option3: Using gstreamer, Named Pipe and opencv
On Linux one may get mjpeg stream and convert it to mpeg4 and feed it to a named pipe. Then read the data from the named pipe in opencv
Step 1. Create Named Pipe
mkfifo stream_fifo
Step 2. Create opencvvideo_test.c
// compile with gcc -ggdb `pkg-config --cflags --libs opencv` opencvvideo_test.c -o opencvvideo_test
#include <stdio.h>
#include "highgui.h"
#include "cv.h"
int main( int argc, char** argv){
IplImage *frame;
int key;
/* supply the AVI file to play */
assert( argc == 2 );
/* load the AVI file */
CvCapture *capture = cvCreateFileCapture(argv[1]) ;//cvCaptureFromAVI( argv[1] );
/* always check */
if( !capture ) return 1;
/* get fps, needed to set the delay */
int fps = ( int )cvGetCaptureProperty( capture, CV_CAP_PROP_FPS );
int frameH = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_HEIGHT);
int frameW = (int) cvGetCaptureProperty(capture, CV_CAP_PROP_FRAME_WIDTH);
/* display video */
cvNamedWindow( "video", CV_WINDOW_AUTOSIZE );
while( key != 'q' ) {
double t1=(double)cvGetTickCount();
/* get a frame */
frame = cvQueryFrame( capture );
double t2=(double)cvGetTickCount();
printf("time: %gms fps: %.2g\n",(t2-t1)/(cvGetTickFrequency()*1000.), 1000./((t2-t1)/(cvGetTickFrequency()*1000.)));
/* always check */
if( !frame ) break;
/* display frame */
cvShowImage( "video", frame );
/* quit if user press 'q' */
key = cvWaitKey( 1000 / fps );
}
/* free memory */
cvReleaseCapture( &capture );
cvDestroyWindow( "video" );
return 0;
}
Step 3. Prepare To Convert From MJPEG to MPEG4 using gstreamer (rate of incoming frames critical)
gst-launch -v souphttpsrc location="http://<ip>/cgi_bin/<mjpeg>.cgi" do-timestamp=true is_live=true ! multipartdemux ! jpegdec ! queue ! videoscale ! 'video/x-raw-yuv, width=640, height=480'! queue ! videorate ! 'video/x-raw-yuv,framerate=30/1' ! queue ! ffmpegcolorspace ! 'video/x-raw-yuv,format=(fourcc)I420' ! ffenc_mpeg4 ! queue ! filesink location=stream_fifo
Step 4. Display Stream in OpenCV
./opencvvideo_test stream_fifo
这篇关于MJPEG流和解码的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!