Windows Media Foundation使用IMFTransform将mp4电影帧解码为2D纹理 [英] Windows Media Foundation using IMFTransform to decode mp4 movie frames to 2D textures
问题描述
我正在尝试使用Windows Media Foundation类对mp4视频进行解码，并将帧转换为2D纹理，DirectX着色器可以使用该纹理进行渲染.我已经能够使用 MFCreateSourceReaderfromURL 读取源流，并且能够读取其主要类型为 MFMEdiaType_Video 而次要类型为的流的媒体类型. MFVideoFormat_H264 如预期.
I'm trying to decode an mp4 video using Windows Media Foundation classes and converting frames in to 2D textures that can be used by a DirectX shader for rendering. I've been able to read the source stream using MFCreateSourceReaderfromURL and been able to read the media type of the stream which has its major type MFMEdiaType_Video and minor type as MFVideoFormat_H264 as expected.
我现在需要将此格式转换为可以用于初始化D3D11_TEXTURE2D资源和资源视图的RGB格式，然后可以将其传递给HLSL像素着色器进行采样.我已经厌倦了使用 IMFTransform 类为我进行转换，但是当我尝试将转换的输出类型设置为任何MFVideoFormat_RGB变体时，我得到了一个错误.我还尝试在源阅读器上设置新的输出类型，然后只是采样以希望以正确的格式采样，但是我再也没有运气了.
I'm now needing to convert this format in to an RGB format that could be used to initialise a D3D11_TEXTURE2D resource and resource view which can then be passed to a HLSL pixel shader for sampling. I've tired using the IMFTransform class to do the conversion for me but when I try to set the output type on the transform to any MFVideoFormat_RGB variant I get an error. I've also tried setting a new output type on the source reader and just Sampling that hoping to get a sample in the correct format but again I've had no luck.
所以我的问题是:
-
这种类型的转换可行吗?
Is this type of conversion possible?
这可以通过IMFTransform/SourceReader类完成，就像我在上面已经累了一样吗，我是否只需要调整代码还是需要手动进行这种类型的转换?
Can this be done through the IMFTransform/SourceReader classes like I've tired above and do I just need to tweak the code or do I need to do this type of conversion manually?
这是将视频纹理数据馈送到着色器以进行采样的最好方法，还是我没有想到过的更简单的选择.
Is this the best way to go about feeding video texture data in to a shader for sampling or is there an easier alternative that I've not thought about.
使用的操作系统是Windows 7，所以我无法使用SourceReaderEx或ID3D11VideoDevice接口，因为据我所知，这些解决方案仅在Windows 8上可用.
The OS being used is Windows 7 so I can't use the SourceReaderEx or ID3D11VideoDevice interface because as far as I'm aware these solutions only seem available on Windows 8.
任何朝着正确方向的帮助/指针将不胜感激，如有必要，我还可以提供一些源代码.
Any help/pointers in the right direction would be greatly appreciated, I can also provide some source code if necessary.
推荐答案
我看到您在理解Media Foundation时有一些错误.您想从MFVideoFormat_H264获取RGB格式的图像，但不要使用解码器H264.您写了我已经厌倦了使用IMFTransform类"-IMFTransform不是类.它是Transform COM对象的接口.您必须创建COM对象Media Foundation H264解码器. Microsoft软件H264解码器的CLSID是CLSID_CMSH264DecoderMFT.但是，您可以从该解码器获得以下格式的输出图像: 输出类型
I see that you have some mistake in understanding of Media Foundation. You want get image in RGB format from MFVideoFormat_H264, but you do not use decoder H264. You wrote "I've tired using the IMFTransform class" - IMFTransform is not class. It is interface for Transform COM objects. You must create COM object Media Foundation H264 decoder. The CLSID for the Microsoft software H264 decoder is CLSID_CMSH264DecoderMFT. However, from that decoder you can get output image in the next formats: Output Types
MFVideoFormat_I420
MFVideoFormat_I420
MFVideoFormat_IYUV
MFVideoFormat_IYUV
MFVideoFormat_NV12
MFVideoFormat_NV12
MFVideoFormat_YUY2
MFVideoFormat_YUY2
MFVideoFormat_YV12
MFVideoFormat_YV12
您可以从其中之一创建D3D11_TEXTURE2D.或者，您可以从我的项目CaptureManager SDK中执行以下操作:
You can create D3D11_TEXTURE2D from one of them. Or you can do something like this from my project CaptureManager SDK:
CComPtrCustom<IMFTransform> lColorConvert;
if (!Result(lColorConvert.CoCreateInstance(__uuidof(CColorConvertDMO))))
{
lresult = MediaFoundationManager::setInputType(
lColorConvert,
0,
lVideoMediaType,
0);
if (lresult)
{
break;
}
DWORD lTypeIndex = 0;
while (!lresult)
{
CComPtrCustom<IMFMediaType> lOutputType;
lresult = lColorConvert->GetOutputAvailableType(0, lTypeIndex++, &lOutputType);
if (!lresult)
{
lresult = MediaFoundationManager::getGUID(
lOutputType,
MF_MT_SUBTYPE,
lSubType);
if (lresult)
{
break;
}
if (lSubType == MFVideoFormat_RGB32)
{
LONG lstride = 0;
MediaFoundationManager::getStrideForBitmapInfoHeader(
lSubType,
lWidth,
lstride);
if (lstride < 0)
lstride = -lstride;
lBitRate = (lHight * (UINT32)lstride * 8 * lNumerator) / lDenominator;
lresult = MediaFoundationManager::setUINT32(
lOutputType,
MF_MT_AVG_BITRATE,
lBitRate);
if (lresult)
{
break;
}
PROPVARIANT lVarItem;
lresult = MediaFoundationManager::getItem(
*aPtrPtrInputMediaType,
MF_MT_FRAME_RATE,
lVarItem);
if (lresult)
{
break;
}
lresult = MediaFoundationManager::setItem(
lOutputType,
MF_MT_FRAME_RATE,
lVarItem);
if (lresult)
{
break;
}
(*aPtrPtrInputMediaType)->Release();
*aPtrPtrInputMediaType = lOutputType.detach();
break;
}
}
}
}
您可以设置ColorConvertDMO，以将H264解码器的输出格式转换为所需的格式.
You can set ColorConvertDMO for converting from output format of the H264 decoder into the needed one of you.
此外，您还可以通过链接查看代码: videoInput .此代码从网络摄像头获取实时视频，并将其解码为RGB.如果在mp4视频文件源上替换网络摄像头源，您将获得接近您需要的解决方案.
Also, you can view code by link: videoInput. This code takes live video from web cam and decode it into the RGB. If you replace web cam source on mp4 video file source you will get the solution which is close to your need.
致谢
这篇关于Windows Media Foundation使用IMFTransform将mp4电影帧解码为2D纹理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
