Mediacodec和相机，颜色空间不正确 [英] Mediacodec and camera, color space incorrect

问题描述

通过引用Aegonis的工作1 和工作2 ，我也得到了H.264流，但颜色不正确。我使用HTC蝴蝶开发。这是我的代码的一部分：

By referring Aegonis's work 1 and work 2, I also got the H.264 stream , but the color is not correct. I am using HTC Butterfly for development. Here is part of my code:

相机：

parameters.setPreviewSize(width, height);
parameters.setPreviewFormat(ImageFormat.YV12);
parameters.setPreviewFrameRate(frameRate);

MediaCodec：

MediaCodec:

mediaCodec = MediaCodec.createEncoderByType("video/avc");
MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 320, 240);
mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 500000);
mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15);
mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);   
mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE); 
mediaCodec.start();   

使用 COLOR_FormatYUV420Planar 时，错误显示[OMX.qcom.video。 encoder.avc]不支持颜色格式19，所以我只能使用 COLOR_FormatYUV420SemiPlanar 。有人知道没有支持的原因吗？

When using COLOR_FormatYUV420Planar the error shows "[OMX.qcom.video.encoder.avc] does not support color format 19," so I can only use "COLOR_FormatYUV420SemiPlanar". Does anyone know the reason why no support?

通过使用：

int colorFormat = 0;
    MediaCodecInfo.CodecCapabilities capabilities = codecInfo.getCapabilitiesForType(mimeType);
    for (int i = 0; i < capabilities.colorFormats.length && colorFormat == 0; i++) {
        int format = capabilities.colorFormats[i];
        Log.e(TAG, "Using color format " + format);           
    }

我们可以使用颜色格式 21 （COLOR_FormatYUV420SemiPlanar）和 2130708361 （没有相应的格式），我认为格式将根据设备而改变。

we can have color format 21 (COLOR_FormatYUV420SemiPlanar) and 2130708361 (no corresponding format), I think the format will change depends on device.

然后，我尝试了从工作1 和工作2 ：

public static byte[] YV12toYUV420PackedSemiPlanar(final byte[] input, final byte[] output, final int width, final int height) {
    /* 
     * COLOR_TI_FormatYUV420PackedSemiPlanar is NV12
     * We convert by putting the corresponding U and V bytes together (interleaved).
     */
    final int frameSize = width * height;
    final int qFrameSize = frameSize/4;

    System.arraycopy(input, 0, output, 0, frameSize); // Y

    for (int i = 0; i < qFrameSize; i++) {
        output[frameSize + i*2] = input[frameSize + i + qFrameSize]; // Cb (U)
        output[frameSize + i*2 + 1] = input[frameSize + i]; // Cr (V)
    }
    return output;
}

public static byte[] YV12toYUV420Planar(byte[] input, byte[] output, int width, int height) {
    /* 
     * COLOR_FormatYUV420Planar is I420 which is like YV12, but with U and V reversed.
     * So we just have to reverse U and V.
     */
    final int frameSize = width * height;
    final int qFrameSize = frameSize/4;

    System.arraycopy(input, 0, output, 0, frameSize); // Y
    System.arraycopy(input, frameSize, output, frameSize + qFrameSize, qFrameSize); // Cr (V)
    System.arraycopy(input, frameSize + qFrameSize, output, frameSize, qFrameSize); // Cb (U)

    return output;
}

public static byte[] swapYV12toI420(byte[] yv12bytes, int width, int height) {
    byte[] i420bytes = new byte[yv12bytes.length];
    for (int i = 0; i < width*height; i++)
        i420bytes[i] = yv12bytes[i];
    for (int i = width*height; i < width*height + (width/2*height/2); i++)
        i420bytes[i] = yv12bytes[i + (width/2*height/2)];
    for (int i = width*height + (width/2*height/2); i < width*height + 2*(width/2*height/2); i++)
        i420bytes[i] = yv12bytes[i - (width/2*height/2)];
    return i420bytes;
}

显然， YV12toYUV420PackedSemiPlanar 的颜色变换效果更好比其他两个。它相对更好，但与真正的颜色相比仍然看起来不同。我的代码有什么问题吗？任何评论将不胜感激。

Obviously, the color transform of YV12toYUV420PackedSemiPlanar performs better than the other two. It is relatively better but still looks different in comparison with the real color. Is there something wrong with my code? Any comment will be appreciated.

推荐答案

阅读结果表明，用于编码各种分辨率的帧
的更一般化的方法是在将帧发送到 MediaCodec 。这是实际的 QualComm （ OMX.qcom.video.encoder.avc ）编码器，我相信HTC蝴蝶，但仍然不能很好地为所有的决议。 720x480 和 176x144 仍然有色度平面根据输出视频未对准。

After reading this discussion it turns out that more generalised way for encoding frames of various resolutions is to align chroma plane by 2048 bytes before sending frame to the MediaCodec. This is actual for QualComm (OMX.qcom.video.encoder.avc) encoder which I believe HTC Butterfly has, but still does not works well for all resolutions. 720x480 and 176x144 are still have chroma plane misaligned according to the output video. Also, avoid resolutions which sizes can't be divided by 16.

转换很简单：

int padding = 0;
if (mediaCodecInfo.getName().contains("OMX.qcom")) {
  padding = (width * height) % 2048;
}
byte[] inputFrameBuffer = new byte[frame.length];
byte[] inputFrameBufferWithPadding = new byte[padding + frame.length];

ColorHelper.NV21toNV12(frame, inputFrameBuffer, width, height);
# copy Y plane
System.arraycopy(inputFrameBuffer, 0, inputFrameBufferWithPadding, 0, inputFrameBuffer.length);
int offset = width * height;
# copy U and V planes aligned by <padding> boundary
System.arraycopy(inputFrameBuffer, offset, inputFrameBufferWithPadding, offset + padding, inputFrameBuffer.length - offset);

这篇关于Mediacodec和相机，颜色空间不正确的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！