从CaptureElement&进行实时视频流处理MediaCapture [英] Perform live video stream processing from CaptureElement & MediaCapture

问题描述

在Windows 8 Store Application中，我倾向于执行实时视频流处理(例如，面部检测).

In my Windows 8 Store Application, I tend to perform live video stream processing (Face detection for example).

在以前的Windows 7应用程序中，我能够使用以下技术(

In my previous Windows 7 application, I was able to use the following technique (Camera Face Detection in C# Using Emgu CV (OpenCV in .NET) and WPF) to perform live video processing. The technique was, having a fixed period timer callback, to actively query image buffer from camera object.

void timer_Tick(object sender, EventArgs e)
{
    Image<Bgr,Byte> currentFrame = capture.QueryFrame();
    // Perform image processing based on currentFrame

---

技术2

我以前在 Android 中使用的另一种技术是，我将在相机本身上安装相机缓冲区预览回调.摄像机会通过传递捕获的摄像机缓冲区来定期触发回调.从那里，我可以执行图像处理.

---

Technique 2

Another technique I had used before in Android is that, I will install a camera buffer preview callback on camera itself. The camera will periodically trigger the callback, by passing along the captured camera buffer. From there, I can perform image processing.

public abstract void onPreviewFrame (byte[] data, Camera camera)
{
    // Perform image processing based on data

---

但是，在有关 Windows 8 Store Application 的视频处理的文章中，它们大多使用内置的处理功能

---

However, in articles which talks about video processing for Windows 8 Store Application, they are mostly using built-in processing functions

• 如何旋转捕获的视频(Windows Store应用使用C#/VB/C ++和XAML)
• 如何为捕获的视频添加视频稳定效果(使用C#/VB/C ++和XAML的Windows应用商店应用

他们都没有演示如何访问原始相机捕获的缓冲区，如何逐像素迭代缓冲区.

我认为我需要利用 CaptureElement&根据

I believe I need to make use of CaptureElement & MediaCapture according to

如果您只想查看网络摄像头输入或毫不费力地将其捕获到文件中.如果您要处理视频实时或覆盖其他一些UI组件–输入...

This method is great if you just want to see the webcam input or capture it to a file with no hassle. If you want to process the video in real time or overlay some other UI components – enter…

CaptureElement&MediaCapture方式

The CaptureElement & MediaCapture Way

但是，在那之后作者不再详细说明.(我希望他这样做:)

However, the author doesn't elaborate more after that. (I hope he does so :)

任何提示如何从CaptureElement&MediaCapture?

Any hint how to perform live video stream processing from CaptureElement & MediaCapture?

推荐答案

首先可以尝试的是MediaCapture.CapturePhotoToStreamAsync，我认为它也仅适用于压缩该照片的场景，但是您可以相当快地将其解码回一个WriteableBitmap并对其像素进行一些处理.不过，您可能不会以这种方式每秒处理太多帧.

What you could try first is MediaCapture.CapturePhotoToStreamAsync which I think also only scenarios where that photo will be compressed, but you could probably fairly quickly decode it back into a WriteableBitmap and do some processing on its pixels. You probably aren't going to process too many frames per second that way though.

更好的方法可能是使用MediaCapture.AddEffectAsync()并用C ++编写一个Media Foundation Transform(MFT)，以处理视频并将结果以某种方式传达回应用程序.

The better way to do it would probably be to use MediaCapture.AddEffectAsync() and write a Media Foundation Transform (MFT) in C++ that would process the video and somehow communicate the results back to the app.

这篇关于从CaptureElement&进行实时视频流处理MediaCapture的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！