使用 Android MediaCodec 从相机编码 H.264 [英] Encoding H.264 from camera with Android MediaCodec

查看:42
本文介绍了使用 Android MediaCodec 从相机编码 H.264的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在尝试让它在 Android 4.1 上运行(使用升级的 Asus Transformer 平板电脑).感谢 Alex 对我上一个问题的回答,我已经能够将一些原始 H.264 数据写入文件,但是此文件只能使用 ffplay -f h264 播放,而且似乎丢失了有关帧率的所有信息(播放速度极快).颜色空间看起来也不正确(atm 在编码器端使用相机的默认设置).

I'm trying to get this to work on Android 4.1 (using an upgraded Asus Transformer tablet). Thanks to Alex's response to my previous question, I already was able to write some raw H.264 data to a file, but this file is only playable with ffplay -f h264, and it seems like it's lost all information regarding the framerate (extremely fast playback). Also the color-space looks incorrect (atm using the camera's default on encoder's side).

public class AvcEncoder {

private MediaCodec mediaCodec;
private BufferedOutputStream outputStream;

public AvcEncoder() { 
    File f = new File(Environment.getExternalStorageDirectory(), "Download/video_encoded.264");
    touch (f);
    try {
        outputStream = new BufferedOutputStream(new FileOutputStream(f));
        Log.i("AvcEncoder", "outputStream initialized");
    } catch (Exception e){ 
        e.printStackTrace();
    }

    mediaCodec = MediaCodec.createEncoderByType("video/avc");
    MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 320, 240);
    mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 125000);
    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15);
    mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar);
    mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
    mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    mediaCodec.start();
}

public void close() {
    try {
        mediaCodec.stop();
        mediaCodec.release();
        outputStream.flush();
        outputStream.close();
    } catch (Exception e){ 
        e.printStackTrace();
    }
}

// called from Camera.setPreviewCallbackWithBuffer(...) in other class
public void offerEncoder(byte[] input) {
    try {
        ByteBuffer[] inputBuffers = mediaCodec.getInputBuffers();
        ByteBuffer[] outputBuffers = mediaCodec.getOutputBuffers();
        int inputBufferIndex = mediaCodec.dequeueInputBuffer(-1);
        if (inputBufferIndex >= 0) {
            ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
            inputBuffer.clear();
            inputBuffer.put(input);
            mediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, 0, 0);
        }

        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
        int outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo,0);
        while (outputBufferIndex >= 0) {
            ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
            byte[] outData = new byte[bufferInfo.size];
            outputBuffer.get(outData);
            outputStream.write(outData, 0, outData.length);
            Log.i("AvcEncoder", outData.length + " bytes written");

            mediaCodec.releaseOutputBuffer(outputBufferIndex, false);
            outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo, 0);

        }
    } catch (Throwable t) {
        t.printStackTrace();
    }

}

将编码器类型更改为video/mp4"显然可以解决帧率问题,但由于主要目标是制作流媒体服务,因此这不是一个好的解决方案.

Changing the encoder type to "video/mp4" apparently solves the framerate-problem, but since the main goal is to make a streaming service, this is not a good solution.

我知道考虑到 SPS 和 PPS NALU,我删除了 Alex 的一些代码,但我希望这不是必需的,因为该信息也来自 outData,我认为编码器会正确格式化.如果不是这种情况,我应该如何在我的文件/流中安排不同类型的 NALU?

I'm aware that I dropped some of Alex' code considering the SPS and PPS NALU's, but I was hoping this would not be necessary since that information was also coming from outData and I assumed the encoder would format this correctly. If this is not the case, how should I arrange the different types of NALU's in my file/stream?

那么,为了制作有效的 H.264 流,我在这里缺少什么?我应该使用哪些设置来匹配相机的色彩空间和编码器的色彩空间?

So, what am I missing here in order to make a valid, working H.264 stream? And which settings should I use to make a match between the camera's colorspace and the encoder's colorspace?

我感觉这更像是一个与 H.264 相关的问题,而不是一个 Android/MediaCodec 主题.还是我仍然没有正确使用 MediaCodec API?

I have a feeling this is more of a H.264-related question than a Android/MediaCodec topic. Or am I still not using the MediaCodec API correctly?

提前致谢.

推荐答案

对于您的快速播放 - 帧速率问题,您无需在此做任何事情.由于它是流媒体解决方案,因此必须提前告知另一方帧速率或每帧的时间戳.这两者都不是基本流的一部分.要么选择预先确定的帧速率,要么传递一些 sdp 或类似的东西,或者使用现有的协议,如 rtsp.在第二种情况下,时间戳是以 rtp 之类的形式发送的流的一部分.然后客户端必须支付 rtp 流并播放它.这就是基本流的工作原理.[如果您有固定速率编码器或提供时间戳,请修复您的帧速率]

For your fast playback - frame rate issue, there is nothing you have to do here. Since it is a streaming solution the other side has to be told the frame rate in advance or timestamps with each frame. Both of these are not part of elementary stream. Either pre-determined framerate is chosen or you pass on some sdp or something like that or you use existing protocols like rtsp. In the second case the timestamps are part of the stream sent in form of something like rtp. Then the client has to depay the rtp stream and play it bacl. This is how elementary streaming works. [either fix your frame rate if you have a fixed rate encoder or give timestamps]

本地 PC 播放会很快,因为它不知道 fps.通过在输入之前给出 fps 参数,例如

Local PC playback will be fast because it will not know the fps. By giving the fps parameter before the input e.g

ffplay -fps 30 in.264

您可以在 PC 上控制播放.

you can control the playback on the PC.

至于不能播放的文件:是否有SPS和PPS.您还应该启用 NAL 标头 - 附件 b 格式.我对 android 了解不多,但是当 h.264 基本流不在任何容器中并且需要转储并稍后播放时,这是可播放的要求.如果 android 默认是 mp4,但默认的 Annexb 标题将被关闭,所以也许有一个开关来启用它.或者,如果您要逐帧获取数据,只需自己添加即可.

As for the file not being playable: Does it have a SPS and PPS. Also you should have NAL headers enabled - annex b format. I don't know much about android, but this is requirement for any h.264 elementary stream to be playable when they are not in any containers and need to be dumped and played later. If android default is mp4, but default annexb headers will be switched off, so perhaps there is a switch to enable it. Or if you are getting data frame by frame, just add it yourself.

至于颜色格式:我猜默认应该可以工作.所以尽量不要设置它.如果没有尝试 422 Planar 或 UVYV/VYUY 交错格式.通常相机就是其中之一.(但不是必须的,这些可能是我经常遇到的).

As for color format: I would guess the default should work. So try not setting it. If not try 422 Planar or UVYV / VYUY interleaved formats. usually cameras are one of those. (but not necessary, these may be the ones I have encountered more often).

这篇关于使用 Android MediaCodec 从相机编码 H.264的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆