如何使用 SharpDX 从网络摄像头捕获帧 [英] How to capture frames from a webcam with SharpDX

问题描述

我正在尝试实现一个网络摄像头捕获应用程序，该应用程序应该拍摄静止帧，将它们显示在屏幕上并保存到磁盘.

由于我已经在使用 SharpDX 来捕获屏幕，我认为使用该库会很好.我不确定 SharpDX 是否有任何视频捕获功能，所以我开始搜索并找到了它看起来像网络摄像头捕获原型的部分:

var attributes = new MediaAttributes(1);attributes.Set(CaptureDeviceAttributeKeys.SourceType, CaptureDeviceAttributeKeys.SourceTypeVideoCapture.Guid);var activates = MediaFactory.EnumDeviceSources(attributes);var dic = new Dictionary();foreach (var activate in activates){var uid = activate.Get(CaptureDeviceAttributeKeys.SourceTypeVidcapSymbolicLink);dic.Add(uid, activate);}var camera = dic.First().Value;

它输出带有奇怪 uid 的 camera.我不确定它是否正确.

在这之后我应该做什么?

编辑

我得到了这段代码的工作.我还是不明白为什么输出很奇怪.

var attributes = new MediaAttributes(1);attributes.Set(CaptureDeviceAttributeKeys.SourceType.Guid, CaptureDeviceAttributeKeys.SourceTypeVideoCapture.Guid);var mediaSource = MediaFactory.EnumDeviceSources(attributes)[0].ActivateObject();mediaSource.CreatePresentationDescriptor(out var presentationDescriptor);var reader = new SourceReader(mediaSource);var mediaTypeIndex = 0;整数宽度，高度；使用 (var mt = reader.GetNativeMediaType(0, mediaTypeIndex)){UnpackLong(mt.Get(MediaTypeAttributeKeys.FrameSize), out width, out height);UnpackLong(mt.Get(MediaTypeAttributeKeys.FrameRate), out var frameRateNumerator, out var frameRateDenominator);UnpackLong(mt.Get(MediaTypeAttributeKeys.PixelAspectRatio), out var aspectRatioNumerator, out var aspectRatioDenominator);}var sample = reader.ReadSample(SourceReaderIndex.AnyStream, SourceReaderControlFlags.None, out var readStreamIndex, out var readFlags, out var timestamp);如果(样本==空)sample = reader.ReadSample(SourceReaderIndex.AnyStream, SourceReaderControlFlags.None, out readStreamIndex, out readFlags, out timestamp);var sourceBuffer = sample.GetBufferByIndex(0);//sample.ConvertToContiguousBuffer();var sourcePointer = sourceBuffer.Lock(out var maxLength, out var currentLength);var data = new byte[sample.TotalLength];Marshal.Copy(sourcePointer, data, 0, sample.TotalLength);var newData = 新字节[宽度 * 4 * 高度];var partWidth = 宽度/4;var partHeight = 高度/3;for (var i = 0; i BGRA = 4新数据[i] = 数据[i + 3];新数据[i + 1] = 数据[i + 2];新数据[i + 2] = 数据[i + 1];新数据[i + 3] = 255；//数据[i];}//var source = BitmapSource.Create(width, height, 96, 96, PixelFormats.Bgra32, null, data, ((width * 24 + 31)/32) * 4);var source = BitmapSource.Create(width, height, 96, 96, PixelFormats.Bgra32, null, newData, width * 4);sourceBuffer.Unlock();sourceBuffer.Dispose();

输出图像是这样的(

图像重复4次，每个部分都有一个灰度图像和一个高度减半的彩色版本.图像的三分之二是透明的.

解决方案

您的输出是 NV12，这里有一些示例代码将 nv12 转换为 rgb

 unsafe private static void TransformImage_NV12(IntPtr pDest, int lDestStride, IntPtr pSrc, int lSrcStride, int dwWidthInPixels, int dwHeightInPixels){uint imageWidth = (uint)dwWidthInPixels;uint widthHalf = imageWidth/2;uint imageHeight = (uint)dwHeightInPixels;byte* nv12Data = (byte*)pSrc;字节* rgbData =(字节*)pDest；uint dataSize = imageWidth * imageHeight * 3;for (uint y = 0; y < imageHeight; y++){for (uint x = 0; x 

I'm trying to implement a webcam capture app which should take still frames, display them on the screen and save to the disk.

Since I'm using SharpDX already to capture the screen, I thought it would be nice to use that library. I was not sure if SharpDX had any video capture capabilities, so I started searching and found parts of what it looks like a webcam capture prototype:

var attributes = new MediaAttributes(1);
attributes.Set<Guid>(CaptureDeviceAttributeKeys.SourceType, CaptureDeviceAttributeKeys.SourceTypeVideoCapture.Guid);
var activates = MediaFactory.EnumDeviceSources(attributes);

var dic = new Dictionary<string, Activate>();
foreach (var activate in activates)
{
    var uid = activate.Get(CaptureDeviceAttributeKeys.SourceTypeVidcapSymbolicLink);
    dic.Add(uid, activate);
}

var camera = dic.First().Value;

It outputs camera with a strange uid. I'm not sure if it's correct.

What I am supposed to do after this?

Edit

I got this code kind of working. I still don't understand why the output is strange.

var attributes = new MediaAttributes(1);
attributes.Set(CaptureDeviceAttributeKeys.SourceType.Guid, CaptureDeviceAttributeKeys.SourceTypeVideoCapture.Guid);

var mediaSource = MediaFactory.EnumDeviceSources(attributes)[0].ActivateObject<MediaSource>();
mediaSource.CreatePresentationDescriptor(out var presentationDescriptor);

var reader = new SourceReader(mediaSource);
var mediaTypeIndex = 0;

int width, height;

using (var mt = reader.GetNativeMediaType(0, mediaTypeIndex))
{
    UnpackLong(mt.Get(MediaTypeAttributeKeys.FrameSize), out  width, out  height);
    UnpackLong(mt.Get(MediaTypeAttributeKeys.FrameRate), out var frameRateNumerator, out var frameRateDenominator);
    UnpackLong(mt.Get(MediaTypeAttributeKeys.PixelAspectRatio), out var aspectRatioNumerator, out var aspectRatioDenominator);
}

var sample = reader.ReadSample(SourceReaderIndex.AnyStream, SourceReaderControlFlags.None, out var readStreamIndex, out var readFlags, out var timestamp);

if (sample == null)
    sample = reader.ReadSample(SourceReaderIndex.AnyStream, SourceReaderControlFlags.None, out readStreamIndex, out readFlags, out timestamp);


var sourceBuffer = sample.GetBufferByIndex(0); // sample.ConvertToContiguousBuffer();
var sourcePointer = sourceBuffer.Lock(out var maxLength, out var currentLength);

var data = new byte[sample.TotalLength];
Marshal.Copy(sourcePointer, data, 0, sample.TotalLength);

var newData = new byte[width * 4 * height];

var partWidth = width / 4;
var partHeight = height / 3;

for (var i = 0; i < sample.TotalLength; i += 4)
{
    //X8R8B8G8 -> BGRA = 4
    newData[i] = data[i + 3];
    newData[i + 1] = data[i + 2];
    newData[i + 2] = data[i + 1];
    newData[i + 3] = 255; //data[i];
}

//var source = BitmapSource.Create(width, height, 96, 96, PixelFormats.Bgra32, null, data, ((width * 24 + 31) / 32) * 4);
var source = BitmapSource.Create(width, height, 96, 96, PixelFormats.Bgra32, null, newData, width * 4);

sourceBuffer.Unlock();
sourceBuffer.Dispose();

The output image is this (I was showing a color spectrum to my webcam):

The image is repeating 4 times, each part has a grayscale image and a color version with half the height. Two thirds of the image is transparent.

解决方案

your output is NV12, here's some sample code to convert nv12 to rgb

    unsafe private static void TransformImage_NV12(IntPtr pDest, int lDestStride, IntPtr pSrc, int lSrcStride, int dwWidthInPixels, int dwHeightInPixels)
    {
        uint imageWidth = (uint)dwWidthInPixels;
        uint widthHalf = imageWidth / 2;
        uint imageHeight = (uint)dwHeightInPixels;

        byte* nv12Data = (byte*)pSrc;
        byte* rgbData = (byte*)pDest;

        uint dataSize = imageWidth * imageHeight * 3;

        for (uint y = 0; y < imageHeight; y++)
        {
            for (uint x = 0; x < imageWidth; x++)
            {
                uint xEven = x & 0xFFFFFFFE;
                uint yEven = y & 0xFFFFFFFE;
                uint yIndex = y * imageWidth + x;
                uint cIndex = imageWidth * imageHeight + yEven * widthHalf + xEven;

                byte yy = nv12Data[yIndex];
                byte cr = nv12Data[cIndex + 0];
                byte cb = nv12Data[cIndex + 1];

                uint outputIndex = (dataSize - (y * imageWidth + x) * 3) - 3;

                rgbData[outputIndex + 0] = (byte)Math.Min(Math.Max((yy + 1.402 * (cr - 128)), 0), 255);
                rgbData[outputIndex + 1] = (byte)Math.Min(Math.Max((yy - 0.344 * (cb - 128) - 0.714 * (cr - 128)), 0), 255);
                rgbData[outputIndex + 2] = (byte)Math.Min(Math.Max((yy + 1.772 * (cb - 128)), 0), 255);
            }
        }
    }

这篇关于如何使用 SharpDX 从网络摄像头捕获帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！