如何获取 MediaCodec 编码器的步幅和 Y 平面对齐值 [英] How to get stride and Y plane alignment values for MediaCodec encoder

问题描述

有一些关于这个主题的相关问题和讨论:

There's a few related questions and discussions on this subject:

• Mediacodec 和相机，色彩空间不正确
• 通过 MediaCodec API 使 QualComm 编码器工作
• https://groups.google.com/d/msg/android-platform/awaNwgb6EbY/a-YiIOwaL0QJ
• https://code.google.com/p/android/issues/detail?id=37769

我正在将相机预览帧(NV21 转换为 NV12)提供给 MediaCodec 编码器(NV12 aka <代码>COLOR_FormatYUV420SemiPlanar).看起来在某些带有 QualComm 编码器的设备上运行的 Android 版本低于 4.3 我必须进行一些输入帧处理才能接收具有正确颜色的后帧.

I am feeding camera preview frames (NV21 converted to NV12) to MediaCodec encoder (NV12 aka COLOR_FormatYUV420SemiPlanar). It looks like that on some devices with QualComm encoder which are running Android versions less than 4.3 I have to do some input frames processing in order to receive back frame with correct color.

在运行 Android 4.2.2 的 Sony Xperia ZR 上，我必须添加 Y 平面对齐，以使其在几乎所有分辨率下都能工作.上面的代码为不能被32分割的宽度添加1024字节对齐，其他分辨率添加2048字节对齐.它使 MediaCodec 能够为所有可以被 16 划分的分辨率正确编码帧(除了 176x144，对于 UV平面看起来没有对齐).

On Sony Xperia ZR running Android 4.2.2 I have to add Y plane alignment in order to make it working on almost all resolutions. Code above adds 1024 bytes alignment for widths which can not be divided by 32 and 2048 bytes alignment for other resolutions. It makes MediaCodec to encode frames properly for all resolutions which can be divided by 16 (except 176x144 for which UV plane looks misaligned).

int getYPadding() {
    if (mediaCodecInfo.getName().contains("OMX.qcom") && android.os.Build.VERSION.SDK_INT < 18) {
        if ((getWidth() % 32) != 0) {
            return (getWidth()*getHeight()) % 1024;
        } else {
            return (getWidth()*getHeight()) % 2048;
        }
    }
    return 0;
}

我已尝试在运行相同 Android 4.2.2 并具有 QualComm 编码器的 LG G2 上测试此对齐方式，并且它看起来它不能正常工作.UV 平面未对齐(框架底部的绿色条纹).我无法计算适用于两部手机的填充.

I've tried to test this alignment on LG G2 which is running same Android 4.2.2 and has QualComm encoder, and it looks like it does not working on it correctly. UV plane is misaligned (a green stripe at the bottom of the frame). I was not able to calculate the padding which will work for both phones.

我还可以访问运行 Android 4.3 和 QualComm 芯片组的 Sony Xperia Z1，它看起来没有这样的问题.每种分辨率的视频看起来都不错，而且 Y 平面无论如何都不需要对齐.

I also have access to Sony Xperia Z1 running Android 4.3 with QualComm chipset and it looks like it does not have such problems. Video on every resolution looks fine and Y plane does not needs to be aligned anyhow.

我知道这与硬件有关并且可能很复杂，但是由于我必须支持在 4.3 之前运行 Android 的用户，所以我有一个问题.是否可以以编程方式确定编码器对于给定颜色格式所期望的 Y 平面对齐和垂直/水平步幅值?

I understand that it's hardware-related and might be complicated, but since I have to support users running Android prior to 4.3 I have a question. Is it possible to programmatically determine Y plane alignment and vertical / horizontal stride values which encoder is expecting for the given color format?

推荐答案

简而言之:没有CTS 测试视频编码直到 Android 4.3 (API 18).

The problem in a nutshell: there were no CTS tests for video encoding until Android 4.3 (API 18).

因此，MediaCodec 在不同设备上的行为不一致，并且有一些错误未被注意到.EncodeDecodeTest 测试执行您询问的功能，因此您可以可靠地将 YUV 数据提供给 4.3+ 设备(尽管您仍然需要运行时检测它是想要平面还是半平面).

As a result, the behavior of MediaCodec across different devices was inconsistent, and a few bugs went unnoticed. The EncodeDecodeTest tests exercise the functions you're asking about, and as a result you can reliably feed YUV data to a 4.3+ device (though you still have to runtime-detect whether it wants planar or semi-planar).

对于您的具体问题，较旧的 Qualcomm 设备上的 Y 平面需要在 2K 边界处对齐，这与您的代码所做的不太一样.对于 720p 视频，这自然发生 (720*1280 == 450 * 2048)，对于 176x144，您需要调整 1280 以在 26624 而不是 25344 处开始 UV 平面.您需要在缓冲区内设置绝对对齐，而不是固定对齐填充量 -- 使用 uvoffset = (width*height + 2047) &~2047.

For your specific question, the Y plane on older Qualcomm devices needs to be aligned at a 2K boundary, which isn't quite what your code is doing. For 720p video this happens naturally (720*1280 == 450 * 2048), for 176x144 you'd adjust by 1280 to start the UV plane at 26624 instead of 25344. You need to set the absolute alignment within the buffer, not a fixed amount of padding -- use uvoffset = (width*height + 2047) & ~2047.

您需要检测编解码器供应商和 Android 软件版本，如果是 4.3 之前的 Qualcomm，则需要进行此调整.如果您的需求发生变化，并且您可以针对 API 18+，这些问题就会消失.(并且您可以使用 Surface 输入到 MediaCodec，这可以避免 U/V 交换问题，尽管这取决于您的需求，可能没有用.)

You will need to detect the codec vendor and Android software version, and if it's Qualcomm on pre-4.3 you need to make this adjustment. If your requirements change, and you can target API 18+, these issues go away. (And you can use Surface input to MediaCodec, which avoids the U/V swap issue, though depending on your needs that may not be useful.)

这篇关于如何获取 MediaCodec 编码器的步幅和 Y 平面对齐值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！