如何在不复制到 CPU 内存的情况下将 ffmpeg 纹理转换为 Open GL 纹理 [英] How to convert an ffmpeg texture to Open GL texture without copying to CPU memory

问题描述

这个 ffmpeg 示例演示了如何进行硬件解码:https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

This ffmpeg example demonstrates how to do hardware decoding: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

在第 109 行，它执行以下操作:

At line 109 it does this:

/* retrieve data from GPU to CPU */
            if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {

我想避免这种情况，因为这需要时间.因此，我需要一种方法来重用 GPU 内存中的解码视频，以重新进行颜色转换.

I want to avoid this because it takes time. Therefore, I need a way to reuse that decoded video, which is in GPU memory, to redo color conversion.

如何将 GPU 内存中的解码纹理转换为 Open GL 中的纹理，而不像上面的代码那样将其返回到 CPU 内存中?

如果以上都不可行，如何使用open gl在解码后的视频中进行颜色转换?我听说 ffmpeg 支持将 opengl 着色器作为输入传递，所以我想这是可能的.

If the above is not possible, how to do color conversion in the decoded video using open gl? I heard that ffmpeg supports passing opengl shaders as input, so I guess it's possible.

推荐答案

简而言之:很复杂.

根据 ffmpeg 在您的系统上使用的硬件后端，您可能需要执行 DirectX/CUDA/OpenGL 互操作.假设您正在使用 VDPAU 后端并希望将其与 OpenGL 互操作.看起来 ffmpeg 没有以文档化的方式从其公共接口公开该功能.

Depending on the hardware backend that ffmpeg uses on your system you might need to do DirectX/CUDA/OpenGL interop. Let's assume that you're using the VDPAU backend and want to interop it with OpenGL. It looks like ffmpeg does not expose that functionality from its public interface in a documented way.

但是，基于 vdpau_transfer_data_from 实现，您似乎可以从 AVFrame 中检索 VdpVideoSurface 如下:

However, based on the vdpau_transfer_data_from implementation, it seems that you can retrieve the VdpVideoSurface from the AVFrame as follows:

VdpVideoSurface surf = (VdpVideoSurface)(uintptr_t)frame->data[3];

从这里您可以将其从 NV_vdpau_interop 和 NV_vdpau_interop2 扩展创建 OpenGL 纹理.

From here you can pass that to VDPAURegisterVideoSurfaceNV from the NV_vdpau_interop and NV_vdpau_interop2 extensions to create OpenGL textures.

这篇关于如何在不复制到 CPU 内存的情况下将 ffmpeg 纹理转换为 Open GL 纹理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！