在android中解码原始H264流? [英] Decoding Raw H264 stream in android?

问题描述

我有一个项目，要求我在 android 中显示视频流，流是原始 H.264，我正在连接到服务器并将从服务器接收字节流.

I have a project where I have been asked to display a video stream in android, the stream is raw H.264 and I am connecting to a server and will receive a byte stream from the server.

基本上我想知道有没有办法将原始字节发送到 android 中的解码器并将其显示在表面上?

Basically I'm wondering is there a way to send raw bytes to a decoder in android and display it on a surface?

我在 android 4.1 中使用新的 MediaCodec 和 MediaExtractor API 成功解码了包装在 mp4 容器中的 H264，不幸的是，我还没有找到使用这些 API 解码原始 H264 文件或流的方法.

I have been successful in decoding H264 wrapped in an mp4 container using the new MediaCodec and MediaExtractor API in android 4.1, unfortunately I have not found a way to decode a raw H264 file or stream using these API's.

我知道一种方法是编译和使用 FFmpeg，但我更愿意使用可以使用硬件加速的内置方法.我也知道 android 支持 RTSP 流，但这不是一个选项.Android 版本不是问题.

I understand that one way is to compile and use FFmpeg but I'd rather use a built in method that can use HW acceleration. I also understand RTSP streaming is supported in android but this is not an option. Android version is not an issue.

推荐答案

很遗憾，我无法为此提供任何代码，但我会根据我的工作原理尽力解释它.

I can't provide any code for this unfortunately, but I'll do my best to explain it based on how I got it to work.

这里是我如何使用 MediaCodec 类.

So here is my overview of how I got raw H.264 encoded video to work using the MediaCodec class.

使用上面的链接有一个获取解码器设置以及如何使用它的示例，您需要对其进行设置以解码 H264 AVC.

Using the link above there is an example of getting the decoder setup and how to use it, you will need to set it up for decoding H264 AVC.

H.264 的格式是由 NAL Units 组成，每个单元以三个字节的起始前缀开头，值为 0x00、0x00、0x01，每个单元根据第 4 个字节的值有不同的类型在这 3 个起始字节之后.一个 NAL 单元不是视频中的一帧，每一帧由多个 NAL 单元组成.

The format of H.264 is that it’s made up of NAL Units, each starting with a start prefix of three bytes with the values 0x00, 0x00, 0x01 and each unit has a different type depending on the value of the 4th byte right after these 3 starting bytes. One NAL Unit IS NOT one frame in the video, each frame is made up of a number of NAL Units.

基本上，我编写了一个方法来查找每个单独的单元并将其传递给解码器(一个 NAL 单元是起始前缀，之后的任何字节直到下一个起始前缀).

Basically I wrote a method that finds each individual unit and passes it to the decoder (one NAL Unit being the starting prefix and any bytes there after up until the next starting prefix).

现在，如果您有解码 H.264 AVC 的解码器设置，并且有来自解码器的 InputBuffer，那么您就可以开始了.您需要使用 NAL 单元填充此 InputBuffer 并将其传递回解码器，并在流的长度内继续执行此操作.但是，为了完成这项工作，我必须首先向解码器传递一个 SPS(序列参数集)NAL 单元.此单元在起始前缀(第 4 个字节)之后的字节值为 0x67，在某些设备上，除非首先收到此单元，否则解码器会崩溃.基本上直到你找到这个单元，忽略所有其他的 NAL 单元并继续解析流直到你找到这个单元，然后你可以将所有其他单元传递给解码器.

Now if you have the decoder setup for decoding H.264 AVC and have an InputBuffer from the decoder then you are ready to go. You need to fill this InputBuffer with a NAL Unit and pass it back to the decoder and continue doing this for the length of the stream. But, to make this work I had to pass the decoder a SPS (Sequence Parameter Set) NAL Unit first. This unit has a byte value of 0x67 after the starting prefix (the 4th byte), on some devices the decoder would crash unless it received this Unit first. Basically until you find this unit, ignore all other NAL Units and keep parsing the stream until you get this unit, then you can pass all other units to the decoder.

有些设备不需要 SPS，有些需要，但最好先传入.

Some devices didn't need the SPS first and some did, but you are better of passing it in first.

现在，如果您有一个在配置时传递给解码器的表面，那么一旦它为一帧获得足够的 NAL 单元，它就会将其显示在表面上.

Now if you had a surface that you passed to the decoder when you configured it then once it gets enough NAL units for a frame it should display it on the surface.

这篇关于在android中解码原始H264流?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！