从编码H.264摄像机采用Android媒体codeC [英] Encoding H.264 from camera with Android MediaCodec

问题描述

我试图得到这个工作在Android 4.1（使用升级后的华硕变压器平板电脑）。由于亚历克斯的回应我的previous问题，我已经能写一些原始的H.264数据到文件，但该文件只播放与 ffplay -f H264 和它看起来像它失去了对于帧率（极速播放）的所有信息。另外，色彩空间看起来不正确（ATM使用相机的默认连接上codeR的一侧）。

I'm trying to get this to work on Android 4.1 (using an upgraded Asus Transformer tablet). Thanks to Alex's response to my previous question, I already was able to write some raw H.264 data to a file, but this file is only playable with ffplay -f h264, and it seems like it's lost all information regarding the framerate (extremely fast playback). Also the color-space looks incorrect (atm using the camera's default on encoder's side).

public class AvcEncoder {

private MediaCodec mediaCodec;
private BufferedOutputStream outputStream;

public AvcEncoder() { 
    File f = new File(Environment.getExternalStorageDirectory(), "Download/video_encoded.264");
    touch (f);
    try {
        outputStream = new BufferedOutputStream(new FileOutputStream(f));
        Log.i("AvcEncoder", "outputStream initialized");
    } catch (Exception e){ 
        e.printStackTrace();
    }

    mediaCodec = MediaCodec.createEncoderByType("video/avc");
    MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", 320, 240);
    mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 125000);
    mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 15);
    mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar);
    mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
    mediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    mediaCodec.start();
}

public void close() {
    try {
        mediaCodec.stop();
        mediaCodec.release();
        outputStream.flush();
        outputStream.close();
    } catch (Exception e){ 
        e.printStackTrace();
    }
}

// called from Camera.setPreviewCallbackWithBuffer(...) in other class
public void offerEncoder(byte[] input) {
    try {
        ByteBuffer[] inputBuffers = mediaCodec.getInputBuffers();
        ByteBuffer[] outputBuffers = mediaCodec.getOutputBuffers();
        int inputBufferIndex = mediaCodec.dequeueInputBuffer(-1);
        if (inputBufferIndex >= 0) {
            ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
            inputBuffer.clear();
            inputBuffer.put(input);
            mediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, 0, 0);
        }

        MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
        int outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo,0);
        while (outputBufferIndex >= 0) {
            ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
            byte[] outData = new byte[bufferInfo.size];
            outputBuffer.get(outData);
            outputStream.write(outData, 0, outData.length);
            Log.i("AvcEncoder", outData.length + " bytes written");

            mediaCodec.releaseOutputBuffer(outputBufferIndex, false);
            outputBufferIndex = mediaCodec.dequeueOutputBuffer(bufferInfo, 0);

        }
    } catch (Throwable t) {
        t.printStackTrace();
    }

}

更改连接codeR类型为视频/ MP4显然解决了帧率-问题，但由于主要目标是使一个流媒体服务，这不是一个很好的解决方案。

Changing the encoder type to "video/mp4" apparently solves the framerate-problem, but since the main goal is to make a streaming service, this is not a good solution.

我知道，我放弃了一些Alex的code考虑SPS和PPS NALU的，但我希望这不会是必要的，因为这些信息也从 outData ，我承担了EN codeR会正确格式化这个。如果不是这种情况，我应该怎么安排不同类型的NALU的在我的文件/流？

I'm aware that I dropped some of Alex' code considering the SPS and PPS NALU's, but I was hoping this would not be necessary since that information was also coming from outData and I assumed the encoder would format this correctly. If this is not the case, how should I arrange the different types of NALU's in my file/stream?

那么，我在这里丢失为了使有效的工作H.264码流？并设置我应该使用，使相机的色彩和EN codeR的色彩之间的匹配？

So, what am I missing here in order to make a valid, working H.264 stream? And which settings should I use to make a match between the camera's colorspace and the encoder's colorspace?

我有一种感觉，这是更比一款Android /媒体codeC主题的H.264相关的问题。还是我仍然无法正常使用媒体codeC API？

I have a feeling this is more of a H.264-related question than a Android/MediaCodec topic. Or am I still not using the MediaCodec API correctly?

在此先感谢。

推荐答案

有关您的快速回放 - 帧速率的问题，没有什么你在这里做。因为它是一个流解决方案的另一侧具有被告知提前或时间戳与每个帧中的帧速率。这两者都是不基本流的一部分。无论是pre-确定帧率选择，或者你传授一些SDP或类似的东西，或者你使用类似RTSP现有的协议。在第二种情况下时间戳位于的像RTP形式发送的流的一部分。然后，客户端必须depay RTP流和BACL播放它。这就是基本的流媒体作品。 [要么修复你的帧速率，如果你有一个固定汇率连接codeR或提供时间戳]

For your fast playback - frame rate issue, there is nothing you have to do here. Since it is a streaming solution the other side has to be told the frame rate in advance or timestamps with each frame. Both of these are not part of elementary stream. Either pre-determined framerate is chosen or you pass on some sdp or something like that or you use existing protocols like rtsp. In the second case the timestamps are part of the stream sent in form of something like rtp. Then the client has to depay the rtp stream and play it bacl. This is how elementary streaming works. [either fix your frame rate if you have a fixed rate encoder or give timestamps]

本地PC播放会很快，因为它不会知道的FPS。通过输入如之前给出的fps参数

Local PC playback will be fast because it will not know the fps. By giving the fps parameter before the input e.g

ffplay -fps 30 in.264

您可以控制​​在PC上播放。

you can control the playback on the PC.

至于该文件不被播放：是否有一个SPS和PPS。你也应该有NAL标头启用 - 附录B的格式。我不知道很多关于Android，但这是当他们没有任何的容器，需要倾倒，后来玩过任何H.264基本流要播放的要求。 如果Android的默认值是MP4，但是默认annexb头将被关闭，所以可能有一个开关来启用它。或者，如果你得到的数据帧一帧，你来就可以加入。

As for the file not being playable: Does it have a SPS and PPS. Also you should have NAL headers enabled - annex b format. I don't know much about android, but this is requirement for any h.264 elementary stream to be playable when they are not in any containers and need to be dumped and played later. If android default is mp4, but default annexb headers will be switched off, so perhaps there is a switch to enable it. Or if you are getting data frame by frame, just add it yourself.

至于颜色格式：我猜默认应该工作。所以尽量不要设置它。  如果没有尝试422平面或UVYV / VYUY交错格式。通常相机是其中的一个。 （但不是必要的，这些也可以是在我所遇到的更频繁）。

As for color format: I would guess the default should work. So try not setting it. If not try 422 Planar or UVYV / VYUY interleaved formats. usually cameras are one of those. (but not necessary, these may be the ones I have encountered more often).

这篇关于从编码H.264摄像机采用Android媒体codeC的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！