WinRT的从视频中提取流帧 [英] WinRT Extract frames from video stream

问题描述

我解码使用内置摄像头的条形码，我这样做的 capElement.Source.CapturePhotoToStreamAsync 来捕获预览照片。它的工作原理，但冻结的应用程序的一瞬间，感觉很笨拙和马车。

I'm decoding barcodes using the built in camera, I do this with the capElement.Source.CapturePhotoToStreamAsync to capture photos from the preview. it works, but freezes the app for a brief moment, which feels very clumsy and buggy.

所以我想这个在后台同时至少留下一个响应UI在处理照片。

So I'd like to this in background while at least leaving a responsive UI while processing the photos.

到目前为止，我想出了这个以捕获视频流：

So far I came up with this to capture the video stream:

 private async void ScanInBackground()
        {
            bool failedScan = true;

            var stream = new InMemoryRandomAccessStream();

            await  capElement.Source.StartRecordToStreamAsync(MediaEncodingProfile.CreateWmv(VideoEncodingQuality.HD1080p), stream);

            while(failedScan)
            {
                Byte[] bytes = await GetBytesFromStream(stream);
                //How to split the bytes into frames?

                Task.Delay(50);
            }

            Dispatcher.RunAsync(CoreDispatcherPriority.Low,() => StopCap()); 
        }

和此方法来获取从流的字节数：

and this method to get the bytes from the stream:

public static async Task<byte[]> GetBytesFromStream(IRandomAccessStream randomStream)
        {
            var reader = new DataReader(randomStream.GetInputStreamAt(0));
            var bytes = new byte[randomStream.Size];
            try
            {
                await reader.LoadAsync((uint)randomStream.Size); reader.ReadBytes(bytes);
            }
            catch(Exception ex)
            {
                Logger.LogExceptionAsync(ex, "GetBytesFromStream");
            }
            return bytes;
        }

从注释的 ScanInBackground ，你可以看到，我不知道如何分割流成照片/帧。

From the comment at the ScanInBackground, you can see that I have no clue how to split the stream into photos/frames.

推荐答案

有一个微软GitHub的页面上的样品是相关的，虽然他们面向Windows 10.您可能感兴趣的迁移项目中得到这个功能。

There is a sample on the Microsoft github page that is relevant, although they target Windows 10. You may be interested in migrating your project to get this functionality.

的 GetPreviewFrame ：此示例将捕获预览画面，而不是全面的照片。一旦它有一个预览框，它能够读取和编辑它的像素

GetPreviewFrame: This sample will capture preview frames as opposed to full-blown photos. Once it has a preview frame, it can read and edit the pixels on it.

下面是相关的部分：

private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
    // Get information about the preview
    var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;

    // Create the video frame to request a SoftwareBitmap preview frame
    var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);

    // Capture the preview frame
    using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
    {
        // Collect the resulting frame
        SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;

        // Add a simple green filter effect to the SoftwareBitmap
        EditPixels(previewFrame);
    }
}

private unsafe void EditPixels(SoftwareBitmap bitmap)
{
    // Effect is hard-coded to operate on BGRA8 format only
    if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
    {
        // In BGRA8 format, each pixel is defined by 4 bytes
        const int BYTES_PER_PIXEL = 4;

        using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
        using (var reference = buffer.CreateReference())
        {
            // Get a pointer to the pixel buffer
            byte* data;
            uint capacity;
            ((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);

            // Get information about the BitmapBuffer
            var desc = buffer.GetPlaneDescription(0);

            // Iterate over all pixels
            for (uint row = 0; row < desc.Height; row++)
            {
                for (uint col = 0; col < desc.Width; col++)
                {
                    // Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
                    var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;

                    // Read the current pixel information into b,g,r channels (leave out alpha channel)
                    var b = data[currPixel + 0]; // Blue
                    var g = data[currPixel + 1]; // Green
                    var r = data[currPixel + 2]; // Red

                    // Boost the green channel, leave the other two untouched
                    data[currPixel + 0] = b;
                    data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
                    data[currPixel + 2] = r;
                }
            }
        }
    }
}

和声明这个类以外的：

[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
    void GetBuffer(out byte* buffer, out uint capacity);
}

当然，你的项目将允许不安全的代码，这一切工作。

And of course, your project will have to allow unsafe code for all of this to work.

有在样品仔细看看，看看如何让所有的细节。或者说，有一个演练中，您可以观看从相机会近期//构建/会议，其中包括通过一些相机样本演练一点点。

Have a closer look at the sample to see how to get all the details. Or, to have a walkthrough, you can watch the camera session from the recent //build/ conference, which includes a little bit of a walkthrough through some camera samples.

这篇关于WinRT的从视频中提取流帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！