有效抓住从视频像素 [英] Efficiently grabbing pixels from video

问题描述

我要寻找抢了视频文件的图像数据的有效方法。我目前正在测试 FilgraphManagerClass.GetCurrentImage（）从 Interop.QuartzTypeLib 库。这确实是我需要的，但痛苦的缓慢。我需要处理每个视频的所有帧。有什么更好的选择呢我？

要求

• 必须帧精确。 <！ - 非常重要


• 给我的访问解码像素缓冲区（阵列 INT 或字节[] ），最好RGB24或RGB32。


• 的缓冲区可以实时或更快的抓住了。我不需要显示视频，我只需要分析的像素。


• 处理MP4文件（H264 / AAC）。我可以重新包装或帧，如果需要通过AviSynth的服务，但没有retranscoding可以参与。

任何建议将受到欢迎。

某些代码的要求：

  FilgraphManagerClass graphClass =新FilgraphManagerClass（）; 
 graphClass.RenderFile（@C：\tmp\tmp.avs）; 
 
 INT SZ =（graphClass.Width * graphClass.Height + 10）* 4; 
 INT [] =缓冲区新INT [SZ  -  1]; 
  

我再通过每帧步进。我在循环是这样的：

  graphClass.GetCurrentImage（REF SZ，出缓冲器[0]）; 
 // DoStuff（缓冲液）; 
 graphClass.CurrentPosition + = graphClass.AvgTimePerFrame; 
  

解决方案

的 IBasicVideo :: GetCurrentImage 您正在使用的方法基本上是用于快照，并仅在传统模式传统视频渲染工作。也就是说，（一）不准确的时候，它可以让你复制帧或，相反，失去帧;及（b）假设你显示视频

相反，你要建立以下类型的过滤器图：文件来源 - > - >样品采集卡过滤器 - > Null渲染。样品采集卡，一个标准组件，可以提供一个回调，使其调用您与来自通过它的数据帧。

然后你从图中被删除的时钟调用 SetReferenceClock（空）过滤器上的图形，以便它运行尽可能快地（而不是实时）。然后你运行图形和所有的视频​​帧提供给您的回调。

要完成在C＃中的任务，您需要使用 DirectShow.NET库。这是 Capture\DxSnap 样品提供了一个简单的例子如何使用样品采集卡。他们这样做是通过 BufferCB 而不是 SampleCB 和它的作品也很好。其他样本也有使用这种方法。

您会找到其他的代码片段非常接近这个任务：

• 寻求使用DirectShowNet 关键帧 - 使用样品采集卡的


• BufferCB不被SampleGrabber 叫 - 同任务音频部分


• 的如何使用访问音频流DirectShow.NET C＃

关于 MP4 文件，你应该考虑以下几点：

1. 为MPEG-4在Windows中支持有限，而您可能需要安装使文件播放的第三方组件。如果GraphEdit中可以读他们，那么你也可以做到。


2. Windows媒体播放器可能会使用，而且很可能，一个新的API，你应该宁可看的GraphEdit


3. 请务必使用的Win32 / 86 平台上的应用程序，以避免陷入场景，您的应用程序在64位域正在运行，同时支持MP4仅在32位组件存在/安装的库

I am looking for an effective way to grab image data off video files. I am currently testing FilgraphManagerClass.GetCurrentImage() from the Interop.QuartzTypeLib library. This does what I need but is painfully slow. I need to process all frames of each video. What better options do I have?

Requirements

• Must be frame accurate. <-- Very important!
• Gives me access to the decoded pixel buffer (array of int or byte[]), ideally RGB24 or RGB32.
• The buffer can be grabbed in realtime or faster. I do not need to display the video, I only need to analyze the pixels.
• Handle mp4 files (h264/aac). I can rewrap or frame serve via AviSynth if needed but no retranscoding can be involved.

Any suggestions would be welcome.

Some code as requested:

FilgraphManagerClass graphClass = new FilgraphManagerClass();
graphClass.RenderFile(@"C:\tmp\tmp.avs");

int sz = (graphClass.Width * graphClass.Height + 10) * 4;    
int[] buffer = new int[sz - 1];

I am then stepping through each frame. I have something like this in the loop:

graphClass.GetCurrentImage(ref sz, out buffer[0]);
//DoStuff(buffer);
graphClass.CurrentPosition += graphClass.AvgTimePerFrame;

解决方案

IBasicVideo::GetCurrentImage method you are using is basically intended for snapshots, and works with legacy video rendering in legacy modes only. That is, (a) it is NOT time accurate, it can get you duplicate frames or, the opposite, lose frames; and (b) it assumes that you display video.

Instead you want to build a filter graph of the following kind: File Source -> ... -> Sample Grabber Filter -> Null Renderer. Sample Grabber, a standard component, can be provided with a callback so that it calls you with any frame data that comes through it.

Then you remove clock from the graph by calling SetReferenceClock(null) on the filter graph so that it run as fast as possible (as opposed to realtime). Then you Run the graph and all video frames are supplied to your callback.

To accomplish the task in C# you need to use DirectShow.NET library. It's Capture\DxSnap sample provides a brief example how to use Sample Grabber. They do it through BufferCB instead of SampleCB and it works well too. Other samples there are also using this approach.

You will find other code snippets very close to this task:

• Seeking keyframes using DirectShowNet - use of Sample Grabber
• BufferCB not being called by SampleGrabber - same task for audio part
• How to access an audio stream using DirectShow.NET C#

Regarding MP4 files you should take into consideration the following:

1. Support for MPEG-4 is limited in Windows, and you might need third party components installed to make the files playable. If GraphEdit can read them, then you can too.
2. Windows Media Player might be using, and is likely to, a newer API and you should rather look at GraphEdit
3. Be sure to use Win32/x86 platform on your application to avoid running into scenario that your app is running in 64-bit domain, while support for MP4 only exists in 32-bit components/libraries installed

这篇关于有效抓住从视频像素的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！