如何将 Android MediaCodec Surface 连接到 Vulkan [英] How to connect Android MediaCodec Surface to Vulkan

问题描述

我对使用 Android MediaCodec 进行解码以及通过 Surface 将 YUV 馈送到 OpenGL 纹理有很好的理解.我想用 Vulkan 做类似的事情.但是我没有成功找到任何文档或示例代码.

I have a pretty good understanding of decoding with Android MediaCodec and feeding YUV through a Surface into an OpenGL texture. I would like to do something similar with Vulkan. However I have not been successful in finding any documentation or sample code.

我的问题是:我将如何连接以下管道?

My question is: how would I wire up the following pipeline?

MediaCodec Video Decoder ⇨ Surface ⇨ texture ⇨ Vulkan

详情

• 使用 MediaCodec#configure
• Surface 是一个 Android Surface(API 链接、链接到 arch.)

OpenGL 比较

为了比较，在 OpenGL 的情况下，Android Surface 的构造和使用方式如下

For comparison, in OpenGL case an Android Surface is constructed and used like so

 textureId = glGenTextures( &textureId )
 surface = new Surface( new SurfaceTexture( textureId ) )
 mediaCodec.configure( surface )

推荐答案

这目前是不可能的，因为无法从 Vulkan 外部导入内存对象，或任何可以导出 Surface 的 SDK Vulkan 对象.看看VK_KHX_external_memory 和相关的扩展，以了解未来这部分内容的工作方式.

This is currently not possible, as there is no way to import memory objects from outside Vulkan, or any SDK Vulkan object that can export a Surface. Take a look at VK_KHX_external_memory and related extensions for how parts of this might work in the future.

EDIT 2018-05-23:现在可以使用 VK_ANDROID_external_memory_android_hardware_buffer 扩展及其依赖的扩展.您可以使用 AImageReader_newWithUsage() 创建一个与 GPU 采样兼容的 AImageReader.从该 AImageReader 获取 ANativeWindow 并将其用作 AMediaCodec 的输出表面.然后对于您收到的每个图像，获取 AHardwareBuffer 并使用扩展名将其导入到 VkDeviceMemory/VkImage 对中.

EDIT 2018-05-23: This is now possible using the VK_ANDROID_external_memory_android_hardware_buffer extension and the extensions it depends on. You can use AImageReader_newWithUsage() to create an AImageReader compatible with GPU sampling. Get the ANativeWindow from that AImageReader and use it as the AMediaCodec's output surface. Then for each image you receive, get the AHardwareBuffer and import that into a VkDeviceMemory/VkImage pair using the extension.

这篇关于如何将 Android MediaCodec Surface 连接到 Vulkan的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！