DXGI桌面复制：编码帧以通过网络发送 [英] DXGI Desktop Duplication: encoding frames to send them over the network

问题描述

我想写一个应用程序，它将捕获屏幕的视频流并将其发送到远程客户端。我发现在Windows上捕获屏幕的最佳方法是使用 DXGI Desktop Duplication API （从Windows 8起可用）。 Microsoft提供了一个整洁的示例，它将重复的帧流传输到屏幕。现在，我一直在想，什么是最简单的，但仍然是相对快速的方式来编码这些帧并通过网络发送。

I'm trying to write an app which will capture a video stream of the screen and send it to a remote client. I've found out that the best way to capture a screen on Windows is to use DXGI Desktop Duplication API (available since Windows 8). Microsoft provides a neat sample which streams duplicated frames to screen. Now, I've been wondering what is the easiest, but still relatively fast way to encode those frames and send them over the network.

这些帧来自 AcquireNextFrame ，其中包含桌面位图和包含已更新的脏区和移动区域的元数据。从这里，我有几个选项：

The frames come from AcquireNextFrame with a surface that contains the desktop bitmap and metadata which contains dirty and move regions that were updated. From here, I have a couple of options:

1. 从DirectX表面提取位图，然后使用外部库如 ffmpeg 将位图系列编码为H.264，并通过RTSP发送。虽然直接，我担心这个方法将太慢，因为它没有利用任何本机的Windows方法。


2. 从此答案中
a>：将D3D纹理转换为IMFSample并使用MediaFoundation的 SinkWriter 对帧进行编码。我找到了本教程的视频编码，但我还没有找到一种方法立即获得编码帧并发送，而不是将它们全部转储到视频文件。

1. Extract a bitmap from a DirectX surface and then use an external library like ffmpeg to encode series of bitmaps to H.264 and send it over RTSP. While straightforward, I fear that this method will be too slow as it isn't taking advantage of any native Windows methods. Converting D3D texture to a ffmpeg-compatible bitmap seems like unnecessary work.
2. From this answer: convert D3D texture to IMFSample and use MediaFoundation's SinkWriter to encode the frame. I found this tutorial of video encoding, but I haven't yet found a way to immediately get the encoded frame and send it instead of dumping all of them to a video file.

因为我以前没有做过这样的事情，我想问我是否正确的方向。最后，我想要一个简单的，优选低延迟的桌面捕获视频流，我可以从远程设备查看。

Since I haven't done anything like this before, I'm asking if I'm moving in the right direction. In the end, I want to have a simple, preferably low latency desktop capture video stream, which I can view from a remote device.

此外，我想知道是否我可以利用脏和移动区域由桌面复制提供。而不是编码帧，我可以通过网络发送它们，并在客户端进行处理，但这意味着我的客户端必须有DirectX 11.1或更高版本可用，这是不可能的，如果我想流到移动平台。

Also, I'm wondering if I can make use of dirty and move regions provided by Desktop Duplication. Instead of encoding the frame, I can send them over the network and do the processing on the client side, but this means that my client has to have DirectX 11.1 or higher available, which is impossible if I would want to stream to a mobile platform.

推荐答案

您可以使用H264编码的 IMFTransform 界面。从 ID3D11Texture2D 获取IMFSample后，只需将其传递到 IMFTransform :: ProcessInput ，即可从 IMFTransform :: ProcessOutput 获取编码的IMFSample。

You can use IMFTransform interface for H264 encoding. Once you get IMFSample from ID3D11Texture2D just pass it to IMFTransform::ProcessInput and get the encoded IMFSample from IMFTransform::ProcessOutput.

请参阅此编码细节的示例。

Refer this example for encoding details.

获得编码的IMFS样本后，网络。

Once you get the encoded IMFSamples you can send them one by one over the network.

这篇关于DXGI桌面复制：编码帧以通过网络发送的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！