如何将Android MediaCodec Surface连接到Vulkan [英] How to connect Android MediaCodec Surface to Vulkan

问题描述

我对使用Android MediaCodec进行解码并将YUV通过Surface馈入OpenGL纹理有很好的理解.我想对Vulkan做类似的事情.但是，我没有找到任何文档或示例代码.

I have a pretty good understanding of decoding with Android MediaCodec and feeding YUV through a Surface into an OpenGL texture. I would like to do something similar with Vulkan. However I have not been successful in finding any documentation or sample code.

我的问题是:我如何连接以下管道?

My question is: how would I wire up the following pipeline?

MediaCodec Video Decoder ⇨ Surface ⇨ texture ⇨ Vulkan

详细信息

• 使用链接到API ，链接到arch.)

• Video decoder is configured using MediaCodec#configure
• Surface is an Android Surface (link to API, link to arch.)

OpenGL比较

为进行比较，在OpenGL情况下，Android Surface的构建和使用方式如下

For comparison, in OpenGL case an Android Surface is constructed and used like so

 textureId = glGenTextures( &textureId )
 surface = new Surface( new SurfaceTexture( textureId ) )
 mediaCodec.configure( surface )

推荐答案

当前无法实现，因为无法从Vulkan外部导入内存对象，也无法从任何可以导出Surface的SDK Vulkan对象导入内存对象.看看 VK_KHX_external_memory 和相关扩展，以了解将来部分内容的工作方式.

This is currently not possible, as there is no way to import memory objects from outside Vulkan, or any SDK Vulkan object that can export a Surface. Take a look at VK_KHX_external_memory and related extensions for how parts of this might work in the future.

编辑2018-05-23:现在可以使用 VK_ANDROID_external_memory_android_hardware_buffer 扩展及其依赖的扩展.您可以使用AImageReader_newWithUsage创建与GPU采样兼容的AImageReader.从该AImageReader获取ANativeWindow并将其用作AMediaCodec的输出表面.然后，对于收到的每个图像，获取AHardwareBuffer并将其使用扩展名导入到VkDeviceMemory/VkImage对中.

EDIT 2018-05-23: This is now possible using the VK_ANDROID_external_memory_android_hardware_buffer extension and the extensions it depends on. You can use AImageReader_newWithUsage to create an AImageReader compatible with GPU sampling. Get the ANativeWindow from that AImageReader and use it as the AMediaCodec's output surface. Then for each image you receive, get the AHardwareBuffer and import that into a VkDeviceMemory/VkImage pair using the extension.

这篇关于如何将Android MediaCodec Surface连接到Vulkan的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！