如何在不复制到CPU内存的情况下将ffmpeg纹理转换为Open GL纹理 [英] How to convert an ffmpeg texture to Open GL texture without copying to CPU memory

问题描述

此ffmpeg示例演示了如何进行硬件解码: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

This ffmpeg example demonstrates how to do hardware decoding: https://github.com/FFmpeg/FFmpeg/blob/release/4.2/doc/examples/hw_decode.c

在109行，它是这样做的:

At line 109 it does this:

/* retrieve data from GPU to CPU */
            if ((ret = av_hwframe_transfer_data(sw_frame, frame, 0)) < 0) {

我想避免这种情况，因为这需要时间.因此，我需要一种方法来重用GPU内存中的已解码视频以重做颜色转换.

I want to avoid this because it takes time. Therefore, I need a way to reuse that decoded video, which is in GPU memory, to redo color conversion.

如何将GPU内存中的解码纹理转换为Open GL中的纹理而又不像上面的代码那样将其返回到CPU内存中?

如果以上操作均不可行，如何使用open gl在解码后的视频中进行颜色转换?我听说ffmpeg支持将opengl着色器作为输入传递，所以我想这是可能的.

If the above is not possible, how to do color conversion in the decoded video using open gl? I heard that ffmpeg supports passing opengl shaders as input, so I guess it's possible.

推荐答案

简而言之:这很复杂.

取决于ffmpeg在系统上使用的硬件后端，您可能需要执行DirectX/CUDA/OpenGL互操作.假设您使用的是VDPAU后端，并且想与OpenGL互操作. ffmpeg似乎没有以书面记录的方式从其公共接口中公开该功能.

Depending on the hardware backend that ffmpeg uses on your system you might need to do DirectX/CUDA/OpenGL interop. Let's assume that you're using the VDPAU backend and want to interop it with OpenGL. It looks like ffmpeg does not expose that functionality from its public interface in a documented way.

但是，基于 vdpau_transfer_data_from 实现，您似乎可以从AVFrame中检索VdpVideoSurface，如下所示:

However, based on the vdpau_transfer_data_from implementation, it seems that you can retrieve the VdpVideoSurface from the AVFrame as follows:

VdpVideoSurface surf = (VdpVideoSurface)(uintptr_t)frame->data[3];

您可以在此处从 NV_vdpau_interop传递给VDPAURegisterVideoSurfaceNV 和 NV_vdpau_interop2 扩展以创建OpenGL纹理.

From here you can pass that to VDPAURegisterVideoSurfaceNV from the NV_vdpau_interop and NV_vdpau_interop2 extensions to create OpenGL textures.

这篇关于如何在不复制到CPU内存的情况下将ffmpeg纹理转换为Open GL纹理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！