AVI，MP4和"Raw"内部的h264 h264流.不同格式的NAL单元(或ffmpeg错误) [英] h264 inside AVI, MP4 and "Raw" h264 streams. Different format of NAL units (or ffmpeg bug)

查看：135
本文介绍了AVI，MP4和"Raw"内部的h264 h264流.不同格式的NAL单元(或ffmpeg错误)的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！

问题描述

TL; DR::我想从AVI/MP4文件中读取原始的h264流，甚至损坏/不完整.

TL;DR: I want to read raw h264 streams from AVI/MP4 files, even broken/incomplete.

关于h264的几乎每个文档都告诉我它由NAL数据包组成.好的.几乎到处都告诉我，数据包应该以00 00 01或00 00 00 01之类的签名开头.例如， https://stackoverflow.com/a/18638298/8167678 ，https://stackoverflow.com/a/17625537/8167678

Almost every document about h264 tells me that it consists of NAL packets. Okay. Almost everywhere told to me that the packet should start with a signature like 00 00 01 or 00 00 00 01. For example, https://stackoverflow.com/a/18638298/8167678, https://stackoverflow.com/a/17625537/8167678

H.264的格式是由NAL单元组成，每个NAL单元均以 起始前缀为三个字节，其值分别为0x00、0x00、0x01 每个单位的类型都不同，具体取决于第4个值 这3个起始字节之后的第一个字节.一个NAL单元不是一帧 在视频中，每个帧都由多个NAL单元组成.

The format of H.264 is that it’s made up of NAL Units, each starting with a start prefix of three bytes with the values 0x00, 0x00, 0x01 and each unit has a different type depending on the value of the 4th byte right after these 3 starting bytes. One NAL Unit IS NOT one frame in the video, each frame is made up of a number of NAL Units.

好的.

我下载了random_youtube_video.mp4并从中删除了一帧:

I downloaded random_youtube_video.mp4 and strip out one frame from it:

ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.avi

得到了: 红色部分-这是AVI容器的一部分，其他部分-实际数据. 如您所见，这里有00 00 24 A9而不是00 00 00 01

And got: Red part - this is part of AVI container, other - actual data. As you can see, here I have 00 00 24 A9 instead of 00 00 00 01

此AVI文件播放完美

我对mp4容器也是如此:

I do same for mp4 container:

如您所见，这里的字节数完全相同. 此MP4文件播放完美

As you can see, here exact same bytes. This MP4 file plays perfectly

我尝试去除原始数据: ffmpeg -i pic.avi -c copy pic.h264

I try to strip out raw data: ffmpeg -i pic.avi -c copy pic.h264

此文件无法在VLC中播放，甚至生成该文件的ffmpeg也无法对其进行解析:

This file can't play in VLC or even ffmpeg, which produced this file, can't parse it:

我下载了mp4流分析器，并得到:

I downloaded mp4 stream analyzer and got:

MP4Box告诉我:

 Cannot find H264 start code
 Error importing pic.h264: BitStream Not Compliant

什么都不起作用，很难学习h264的内部原理.

It very hard to learn internals of h264, when nothing works.

所以，我有问题:

  1. mp4里面有什么实际数据?
  2. 我必须阅读以解码该数据(我的意思是不同的附件)
  3. 如何从此中断的"原始流中读取流并获取解码的图像(甚至使用ffmpeg)?

更新:

ffmpeg中似乎有错误:

It seems bug in ffmpeg:

当我进行两次转换时:

         ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.mp4
         ffmpeg pic.mp4 -c copy pic.h264

但是当我直接转换文件时:

But when I convert file directly:

ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.h264

我有NAL签名和一个额外的NAL单元.其他字节相同(已选择).

I have NALs signatures and one extra NAL unit. Other bytes are same (selected).

这是虫子吗?

更新

不是，这不是错误，U必须使用-bsf h264_mp4toannexb选项将流另存为附件B"格式(带前缀)

Not, this is not bug, U must use option -bsf h264_mp4toannexb to save stream as "Annex B" format (with prefixes)

推荐答案

我想从AVI文件中读取原始的h264流，甚至损坏/不完整."

几乎到处都告诉我，该数据包应该以如下签名开头:
00 00 01或00 00 00 01"

"...如您所见，这里有00 00 24 A9而不是00 00 00 01"

"...As you can see, here I have 00 00 24 A9 instead of 00 00 00 01"

您的H264为AVCC格式，这意味着它使用数据 size (而不是数据起始代码).只有附件B会将您提到的签名作为起始代码.

Your H264 is in AVCC format which means it uses data sizes (instead of data start codes). It is only Annex-B that will have your mentioned signature as start code.

您查找帧不是通过查找起始代码，而是只是跳过帧大小以达到(请求的)帧的最终正确偏移量...

You seek frames, not by looking for start codes, but instead you just do skipping by frame sizes to reach the final correct offset of a (requested) frame...

AVI处理:

  • 读取大小(四个)字节(32位整数，小尾数).

提取接下来的后续字节，直到最大大小.

Extract the next following bytes up to size amount.

这是您的H.264帧(采用AVCC格式)，对字节进行解码以查看图像.

This is your H.264 frame (in AVCC format), decode the bytes to view image.

要转换为附件B，请尝试用00 00 00 01替换H.264 帧字节的前4个字节.

To convert into Annex-B, try replacing first 4 bytes of H.264 frame bytes with 00 00 00 01.

考虑显示的AVI字节(请参见> 第一 图片):

Consider your shown AVI bytes (see first picture) :

00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00     ................
00 00 00 00 4C 49 53 54 BA 24 00 00 6D 6F 76 69     ....LISTº$..movi
30 30 64 63 AD 24 00 00 00 00 24 A9 65 88 84 27     00dc.$....$©eˆ„'
C7 11 FE B3 C7 83 08 00 08 2A 7B 6E 59 B5 71 E1     Ç.þ³Çƒ...*{nYµqá
E3 9C 0E 73 E7 10 50 00 18 E9 25 F7 AA 7D 9C 30     ãœ.sç.P..é%÷ª}œ0
E6 2F 0F 20 00 3A 64 AA CA 5E 4F CA FF AE 20 04     æ/. .:dªÊ^OÊÿ® .
07 81 40 00 48 00 0A 28 71 21 84 48 06 18 90 0C     ..@.H..(q!„H....
31 14 57 9E 7A CD 63 A0 E0 9B 96 69 C5 18 AE F2     1.WžzÍc à›–iÅ.®ò
E6 07 02 29 01 20 10 70 A1 0F 8C BC 73 F0 78 FA     æ..). .p¡.Œ¼sðxú
9E 1D E1 C2 BF 8C 62 CE CE AC 14 5A A4 E1 45 44     ž.á¿ŒbÎά.Z¤áED
38 38 85 DB 12 57 3E F6 E0 FB AE 03 04 21 62 8D     88…Û.W>öàû®..!b.
F6 F1 1E 37 1C A2 FF 75 1C F1 02 66 0C 92 07 06     öñ.7.¢ÿu.ñ.f.’..
15 7C 90 15 6F 7D FC BD 13 1E 2B 0C 14 3C 0C 00     .|..o}ü½..+..<..
B0 EA 6F 53 B4 98 D7 80 7A 68 3E 34 69 20 D2 FA     °êoS´˜×€zh>4i Òú
F0 91 FC 75 C6 00 01 18 C0 00 3B 9A C5 E2 7D BF     ð‘üuÆ...À.;šÅâ}¿

一些解释:

  • 忽略前导多个00个字节.

4C 49 53 54 D6 3C 00 00 6D 6F 76 69包括30 30 64 63 = AVI列表"标头.

4C 49 53 54 D6 3C 00 00 6D 6F 76 69 including 30 30 64 63 = AVI "List" header.

AD 24 00 00 ==十进制9389是AVI自身的H264大小(必须在 Little Endian 中读取).

AD 24 00 00 == decimal 9389 is AVI's own size of H264 item (must read in Little Endian).

请注意，AVI字节包括...
-记录 item 的总大小(AD 24 00 00 ...或对Little Endian取反:00 00 24 AD) -，后跟 item 数据(00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF).

Notice that the AVI bytes include...
- a note of item's total size (AD 24 00 00... or reverse for Little Endian : 00 00 24 AD)
- followed by item data (00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF).

此 size 既包含AVI的" size "条目的4个字节，又包含 item 自身字节的预期字节长度.可以简单地写为:

This size includes both the 4 bytes of the AVI's"size" entry + expected bytes length of the item's own bytes. Can be written simply as:

AVI_Item_Size = ( 4 + item_H264_Frame.length );

AVI中的H.264视频帧字节:

接下来是 item 数据，它是H.264 视频帧.由于格式/字节布局的巧合，它也保存了 data 的 size 的4个字节的条目(因为您的H264是AVCC格式，如果它是附件- B，那么您将在此处看到起始码字节，而不是大小字节).

Next follows the item data, which is the H.264 video frame. By sheer coincidence of formats/bytes layout, it too holds a 4-byte entry for data's size (since your H264 is in AVCC format, if it was Annex-B then you would be seeing start code bytes here instead of size bytes).

与AVI字节不同，这些H264 大小字节以 Big Endian 格式写入.

Unlike AVI bytes, these H264 size bytes are written in Big Endian format.

  • 00 00 24 A9 =此视频帧的字节大小(而不是起始代码:00 00 00 01).

  • 00 00 24 A9 = size of bytes for this video frame (instead of start code : 00 00 00 01).

65 88 84 27 C7 11 FE B3 C7 = H.264 关键帧(始终以X5开头，其中X值基于其他设置).

65 88 84 27 C7 11 FE B3 C7 = H.264 keyframe (always begins X5, where the X value is based on other settings).

如果后面跟有...，请记住四个大小的字节(甚至是起始代码)后的内容.

Remember after four size bytes (or even start codes) if followed by...

  • 字节X5 =关键帧(IDR)，例如字节65.
  • 字节X1 = P或B帧，例如字节41.
  • 字节X6 = SEI(补充增强信息).
  • 字节X7 = SPS(序列参数集).
  • 字节X8 = PPS(图片参数集).
  • 字节00 00 00 X9 =访问单元定界符.
  • byte X5 = keyframe (IDR), example byte 65.
  • byte X1 = P or B frame, example byte 41.
  • byte X6 = SEI (Supplemental Enhancement Information).
  • byte X7 = SPS (Sequence Parameter Set).
  • byte X8 = PPS (Picture Parameter Set).
  • bytes 00 00 00 X9 = Access unit delimiter.

如果在AVI文件中搜索完全相同的字节，则可以找到H.264.请参见 第三 图片，这些是您的H.264字节(它们被剪切并粘贴到AVI容器中).

You can find the H.264 if you search for exact same bytes within AVI file. See third picture, these are your H.264 bytes (they are cut & pasted into the AVI container).

有时，一帧被切成不同的NAL单元.因此，如果您提取关键帧而只显示1/2或1/3而不是完整图像，则只需抓住下一个或两个NAL，然后重试解码即可.

Sometimes a frame is sliced into different NAL units. So if you extract a key frame and it only shows 1/2 or 1/3 instead of full image, just grab next one or two NAL and re-try the decode.

这篇关于AVI，MP4和"Raw"内部的h264 h264流.不同格式的NAL单元(或ffmpeg错误)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！

查看全文
相关文章
其他开发最新文章
热门教程
热门工具
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆