从内存缓冲区处理视频流 [英] process video stream from memory buffer

问题描述

我需要从专有网络协议（我已经知道如何做）解析视频流（mpeg ts），然后我想使用OpenCV处理视频流到帧。我知道如何使用cv :: VideoCapture从文件或标准URL，但我想设置OpenCV从内存中的缓冲区，我可以存储视频流数据，直到它需要读取。有一种方法来设置回调方法（或任何其他interfrace），使我仍然可以使用cv :: VideoCapture对象？有没有更好的方法来完成处理视频，把它写出来一个文件，然后重新阅读它。我也会招待使用FFMPEG直接，如果这是一个更好的选择。我想我可以将AVFrames转换为Mat如果需要。

I need to parse a video stream (mpeg ts) from proprietary network protocol (which I already know how to do) and then I would like to use OpenCV to process the video stream into frames. I know how to use cv::VideoCapture from a file or from a standard URL, but I would like to setup OpenCV to read from a buffer(s) in memory where I can store the video stream data until it is needed. Is there a way to setup a call back method (or any other interfrace) so that I can still use the cv::VideoCapture object? Is there a better way to accomplish processing the video with out writing it out to a file and then re-reading it. I would also entertain using FFMPEG directly if that is a better choice. I think I can convert AVFrames to Mat if needed.

推荐答案

我最近有类似的需求。我正在寻找一种方法在OpenCV播放已经在内存中的视频，但没有必须写入视频文件到磁盘。我发现，ffmpeg接口已经支持通过 av_open_input_stream 。与在OpenCV中用于打开文件的 av_open_input_file 调用相比，只需要一些更多的准备工作。

I had a similar need recently. I was looking for a way in OpenCV to play a video that was already in memory, but without ever having to write the video file to disk. I found out that the ffmpeg interface already supports this through av_open_input_stream. There is just a little more prep work required compared to the av_open_input_file call used in OpenCV to open a file.

在以下两个网站之间，我能够使用ffmpeg调用组合一个工作解决方案。有关详情，请参阅这些网站上的信息：

Between the following two websites I was able to piece together a working solution using the ffmpeg calls. Please refer to the information on these websites for more details:

http://ffmpeg.arrozcru.org/forum/viewtopic.php?f=8&t=1170

< a href =http://cdry.wordpress.com/2009/09/09/using-custom-io-callbacks-with-ffmpeg/> http://cdry.wordpress.com/2009/09/09/ use-custom-io-callbacks-with-ffmpeg /

为了使其在OpenCV中正常工作，我最后向<$ c添加了一个新函数$ c> CvCapture_FFMPEG class：

To get it working in OpenCV, I ended up adding a new function to the CvCapture_FFMPEG class:

virtual bool openBuffer( unsigned char* pBuffer, unsigned int bufLen );

我在highgui DLL中通过新的API调用提供了访问，类似于 cvCreateFileCapture 。新的 openBuffer 函数基本上与 open（const char * _filename）函数相同，具有以下区别： / p>

I provided access to it through a new API call in the highgui DLL, similar to cvCreateFileCapture. The new openBuffer function is basically the same as the open( const char* _filename ) function with the following difference:

err = av_open_input_file(&ic, _filename, NULL, 0, NULL);

替换为：

ic = avformat_alloc_context();
ic->pb = av_alloc_put_byte(pBuffer, bufLen, 0, pBuffer, read_buffer, NULL, NULL);

if(!ic->pb) {
    // handle error
}

// Need to probe buffer for input format unless you already know it
AVProbeData probe_data;
probe_data.buf_size = (bufLen < 4096) ? bufLen : 4096;
probe_data.filename = "stream";
probe_data.buf = (unsigned char *) malloc(probe_data.buf_size);
memcpy(probe_data.buf, pBuffer, 4096);

AVInputFormat *pAVInputFormat = av_probe_input_format(&probe_data, 1);

if(!pAVInputFormat)
    pAVInputFormat = av_probe_input_format(&probe_data, 0);

// cleanup
free(probe_data.buf);
probe_data.buf = NULL;

if(!pAVInputFormat) {
    // handle error
}

pAVInputFormat->flags |= AVFMT_NOFILE;

err = av_open_input_stream(&ic , ic->pb, "stream", pAVInputFormat, NULL);

此外，请务必调用 av_close_input_stream 在这种情况下， CvCapture_FFMPEG :: close（）函数代替 av_close_input_file 。

Also, make sure to call av_close_input_stream in the CvCapture_FFMPEG::close() function instead of av_close_input_file in this situation.

现在，传递给 av_alloc_put_byte 的 read_buffer 回调函数定义为：

Now the read_buffer callback function that is passed in to av_alloc_put_byte I defined as:

static int read_buffer(void *opaque, uint8_t *buf, int buf_size)
{
    // This function must fill the buffer with data and return number of bytes copied.
    // opaque is the pointer to private_data in the call to av_alloc_put_byte (4th param)

    memcpy(buf, opaque, buf_size);
    return buf_size;
}

此解决方案假设整个视频包含在内存缓冲区中，调整以处理流式传输数据。

This solution assumes the entire video is contained in a memory buffer and would probably have to be tweaked to work with streaming data.

就是这样！ Btw，我使用OpenCV版本2.1，所以YMMV。

So that's it! Btw, I'm using OpenCV version 2.1 so YMMV.

这篇关于从内存缓冲区处理视频流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！