H.264流的序列/图片参数集的可能位置 [英] Possible Locations for Sequence/Picture Parameter Set(s) for H.264 Stream
问题描述
我正在研究H.264解码器，我想知道在哪里可以找到SPS和PPS.我的参考文献告诉我，这些是在H.264-Stream中编码的NAL单元，但是当我使用IsoViewer查看example-MP4-File时，它说SPS和PPS在avcC框中.
I'm working on a H.264 Decoder and I'm wondering where to find the SPS and PPS. My reference literature tells me that those are NAL Units encoded in the H.264-Stream, but when I look into an example-MP4-File with IsoViewer, it says that the SPS and PPS are in the avcC Box.
这是如何工作的?如何查找.mkv文件或其他H.264容器?
How exactly does this work? How does it look for .mkv files or other H.264 containers?
提前谢谢！
推荐答案
首先，重要的是要了解没有单一的标准H.264基本比特流格式.规范文档中确实包含一个附件，特别是附件B，它描述了一种可能的格式，但这并不是实际要求.该标准规定了如何将视频编码为单独的数据包.这些数据包的存储和传输方式对集成商开放.
First off, it's important to understand that there is no single standard H.264 elementary bitstream format. The specification document does contain an Annex, specifically Annex B, that describes one possible format, but it is not an actual requirement. The standard specifies how video is encoded into individual packets. How these packets are stored and transmitted is left open to the integrator.
这些数据包称为网络抽象层单元.通常将每个数据包缩写为NALU(或有时只是NAL)，可以分别对其进行解析和处理.每个NALU的第一个字节都包含NALU类型，特别是位3到位7.(位0始终为OFF，位1-2指示另一个NALU是否引用了NALU).
The packets are called Network Abstraction Layer Units. Often abbreviated NALU (or sometimes just NAL) each packet can be individually parsed and processed. The first byte of each NALU contains the NALU type, specifically bits 3 through 7. (bit 0 is always off, and bits 1-2 indicate whether a NALU is referenced by another NALU).
定义了19种不同的NALU类型，分为两类:VCL和非VCL:
There are 19 different NALU types defined separated into two categories, VCL and non-VCL:
- VCL或视频编码层数据包包含实际的视觉信息.
- 非VCL包含解码视频可能需要或不需要的元数据.
单个NALU或什至是VCL NALU与框架不同.可以将一个框架切片"为多个NALU.就像可以切比萨饼一样.然后将一个或多个切片虚拟分组为包含一帧的访问单元(AU).切片的确会付出少量的质量成本，因此并不经常使用.
A single NALU, or even a VCL NALU is NOT the same thing as a frame. A frame can be ‘sliced’ into several NALUs. Just like you can slice a pizza. One or more slices are then virtually grouped into a Access Units (AU) that contain one frame. Slicing does come at a slight quality cost, so it is not often used.
下面是所有已定义的NALU的表.
Below is a table of all defined NALUs.
0 Unspecified non-VCL
1 Coded slice of a non-IDR picture VCL
2 Coded slice data partition A VCL
3 Coded slice data partition B VCL
4 Coded slice data partition C VCL
5 Coded slice of an IDR picture VCL
6 Supplemental enhancement information (SEI) non-VCL
7 Sequence parameter set non-VCL
8 Picture parameter set non-VCL
9 Access unit delimiter non-VCL
10 End of sequence non-VCL
11 End of stream non-VCL
12 Filler data non-VCL
13 Sequence parameter set extension non-VCL
14 Prefix NAL unit non-VCL
15 Subset sequence parameter set non-VCL
16 Depth parameter set non-VCL
17..18 Reserved non-VCL
19 Coded slice of an auxiliary coded picture without partitioning non-VCL
20 Coded slice extension non-VCL
21 Coded slice extension for depth view components non-VCL
22..23 Reserved non-VCL
24..31 Unspecified non-VCL
有几种NALU类型，对它们的了解可能对以后有所帮助.
There are a couple of NALU types where having knowledge of may be helpful later.
- 序列参数集(SPS)..此非VCL NALU包含配置解码器所需的信息，例如配置文件，级别，分辨率，帧频.
- 图片参数集(PPS).与SPS相似，该非VCL包含有关熵编码模式，片段组，运动预测和解块滤波器的信息.
- 即时解码器刷新(IDR).此VCL NALU是自包含的图像切片.也就是说，无需参考任何其他NALU保存SPS和PPS即可解码和显示IDR.
- 访问单元定界符(AUD). AUD是可选的NALU，可用于定界基本流中的帧.它不是必需的(除非容器/协议(如TS另有说明))，并且通常不包含它是为了节省空间，但是在无需完全解析每个NALU的情况下查找帧的开始会很有用. /li>
- Sequence Parameter Set (SPS). This non-VCL NALU contains information required to configure the decoder such as profile, level, resolution, frame rate.
- Picture Parameter Set (PPS). Similar to the SPS, this non-VCL contains information on entropy coding mode, slice groups, motion prediction and deblocking filters.
- Instantaneous Decoder Refresh (IDR). This VCL NALU is a self contained image slice. That is, an IDR can be decoded and displayed without referencing any other NALU save SPS and PPS.
- Access Unit Delimiter (AUD). An AUD is an optional NALU that can be use to delimit frames in an elementary stream. It is not required (unless otherwise stated by the container/protocol, like TS), and is often not included in order to save space, but it can be useful to finds the start of a frame without having to fully parse each NALU.
NALU不包含其大小.因此，简单地将NALU连接起来以创建流将不起作用，因为您将不知道一个停在哪里，下一个在哪里开始.
A NALU does not contain is its size. Therefore simply concatenating the NALUs to create a stream will not work because you will not know where one stops and the next begins.
附件B规范通过在每个NALU之前要求开始代码"来解决此问题.起始码为2或3个0x00字节，后跟一个0x01字节.例如0x000001或0x00000001.
The Annex B specification solves this by requiring ‘Start Codes’ to precede each NALU. A start code is 2 or 3 0x00 bytes followed with a 0x01 byte. e.g. 0x000001 or 0x00000001.
4字节变量对于通过串行连接进行传输非常有用，因为通过查找31个零位后跟一个1字节来对流进行字节对齐是微不足道的.如果下一位是0(因为每个NALU都以0开头)，则它是NALU的开始. 4字节变化通常仅用于发信号通知流中的随机访问点，例如SPS PPS AUD和IDR，而3字节变化通常用于其他地方以节省空间.
The 4 byte variation is useful for transmission over a serial connection as it is trivial to byte align the stream by looking for 31 zero bits followed by a one. If the next bit is 0 (because every NALU starts with a 0 bit), it is the start of a NALU. The 4 byte variation is usually only used for signaling random access points in the stream such as a SPS PPS AUD and IDR Where as the 3 byte variation is used everywhere else to save space.
开始代码之所以起作用，是因为在非RBSP NALU中四个字节序列0x000000，0x000001，0x000002和0x000003是非法的.因此，在创建NALU时，请小心转义这些值，否则这些值可能会与起始代码混淆.这可以通过插入防止仿真"字节0x03来实现，以使0x000001变为0x00000301.
Start codes work because the four byte sequences 0x000000, 0x000001, 0x000002 and 0x000003 are illegal within a non-RBSP NALU. So when creating a NALU, care is taken to escape these values that could otherwise be confused with a start code. This is accomplished by inserting an ‘Emulation Prevention’ byte 0x03, so that 0x000001 becomes 0x00000301.
解码时，寻找和忽略仿真阻止字节很重要.由于仿真预防字节几乎可以出现在NALU中的任何位置，因此在文档中假定它们已被删除通常更为方便.没有仿真预防字节的表示形式称为原始字节序列有效载荷(RBSP).
When decoding, it is important to look for and ignore emulation prevention bytes. Because emulation prevention bytes can occur almost anywhere within a NALU, it is often more convenient in documentation to assume they have already been removed. A representation without emulation prevention bytes is called Raw Byte Sequence Payload (RBSP).
让我们看一个完整的例子.
Let's look at a complete example.
0x0000 | 00 00 00 01 67 64 00 0A AC 72 84 44 26 84 00 00
0x0010 | 03 00 04 00 00 03 00 CA 3C 48 96 11 80 00 00 00
0x0020 | 01 68 E8 43 8F 13 21 30 00 00 01 65 88 81 00 05
0x0030 | 4E 7F 87 DF 61 A5 8B 95 EE A4 E9 38 B7 6A 30 6A
0x0040 | 71 B9 55 60 0B 76 2E B5 0E E4 80 59 27 B8 67 A9
0x0050 | 63 37 5E 82 20 55 FB E4 6A E9 37 35 72 E2 22 91
0x0060 | 9E 4D FF 60 86 CE 7E 42 B7 95 CE 2A E1 26 BE 87
0x0070 | 73 84 26 BA 16 36 F4 E6 9F 17 DA D8 64 75 54 B1
0x0080 | F3 45 0C 0B 3C 74 B3 9D BC EB 53 73 87 C3 0E 62
0x0090 | 47 48 62 CA 59 EB 86 3F 3A FA 86 B5 BF A8 6D 06
0x00A0 | 16 50 82 C4 CE 62 9E 4E E6 4C C7 30 3E DE A1 0B
0x00B0 | D8 83 0B B6 B8 28 BC A9 EB 77 43 FC 7A 17 94 85
0x00C0 | 21 CA 37 6B 30 95 B5 46 77 30 60 B7 12 D6 8C C5
0x00D0 | 54 85 29 D8 69 A9 6F 12 4E 71 DF E3 E2 B1 6B 6B
0x00E0 | BF 9F FB 2E 57 30 A9 69 76 C4 46 A2 DF FA 91 D9
0x00F0 | 50 74 55 1D 49 04 5A 1C D6 86 68 7C B6 61 48 6C
0x0100 | 96 E6 12 4C 27 AD BA C7 51 99 8E D0 F0 ED 8E F6
0x0110 | 65 79 79 A6 12 A1 95 DB C8 AE E3 B6 35 E6 8D BC
0x0120 | 48 A3 7F AF 4A 28 8A 53 E2 7E 68 08 9F 67 77 98
0x0130 | 52 DB 50 84 D6 5E 25 E1 4A 99 58 34 C7 11 D6 43
0x0140 | FF C4 FD 9A 44 16 D1 B2 FB 02 DB A1 89 69 34 C2
0x0150 | 32 55 98 F9 9B B2 31 3F 49 59 0C 06 8C DB A5 B2
0x0160 | 9D 7E 12 2F D0 87 94 44 E4 0A 76 EF 99 2D 91 18
0x0170 | 39 50 3B 29 3B F5 2C 97 73 48 91 83 B0 A6 F3 4B
0x0180 | 70 2F 1C 8F 3B 78 23 C6 AA 86 46 43 1D D7 2A 23
0x0190 | 5E 2C D9 48 0A F5 F5 2C D1 FB 3F F0 4B 78 37 E9
0x01A0 | 45 DD 72 CF 80 35 C3 95 07 F3 D9 06 E5 4A 58 76
0x01B0 | 03 6C 81 20 62 45 65 44 73 BC FE C1 9F 31 E5 DB
0x01C0 | 89 5C 6B 79 D8 68 90 D7 26 A8 A1 88 86 81 DC 9A
0x01D0 | 4F 40 A5 23 C7 DE BE 6F 76 AB 79 16 51 21 67 83
0x01E0 | 2E F3 D6 27 1A 42 C2 94 D1 5D 6C DB 4A 7A E2 CB
0x01F0 | 0B B0 68 0B BE 19 59 00 50 FC C0 BD 9D F5 F5 F8
0x0200 | A8 17 19 D6 B3 E9 74 BA 50 E5 2C 45 7B F9 93 EA
0x0210 | 5A F9 A9 30 B1 6F 5B 36 24 1E 8D 55 57 F4 CC 67
0x0220 | B2 65 6A A9 36 26 D0 06 B8 E2 E3 73 8B D1 C0 1C
0x0230 | 52 15 CA B5 AC 60 3E 36 42 F1 2C BD 99 77 AB A8
0x0240 | A9 A4 8E 9C 8B 84 DE 73 F0 91 29 97 AE DB AF D6
0x0250 | F8 5E 9B 86 B3 B3 03 B3 AC 75 6F A6 11 69 2F 3D
0x0260 | 3A CE FA 53 86 60 95 6C BB C5 4E F3
这是一个包含3个NALU的完整AU.如您所见，我们从开始代码开始，然后是SPS(SPS以67开头).在SPS内，您将看到两个Emulation Prevention字节.没有这些字节，非法序列0x000000将出现在这些位置.接下来，您将看到一个起始代码，后跟一个PPS(PPS以68开头)和一个最终的起始代码，后跟一个IDR分片.这是完整的H.264流.如果您将这些值输入到十六进制编辑器中，并以.264扩展名保存文件，则可以将其转换为以下图像:
This is a complete AU containing 3 NALUs. As you can see, we begin with a Start code followed by an SPS (SPS starts with 67). Within the SPS, you will see two Emulation Prevention bytes. Without these bytes the illegal sequence 0x000000 would occur at these positions. Next you will see a start code followed by a PPS (PPS starts with 68) and one final start code followed by an IDR slice. This is a complete H.264 stream. If you type these values into a hex editor and save the file with a .264 extension, you will be able to convert it to this image:
附件B通常以直播和流格式使用，例如传输流，空中广播和DVD.在这些格式中，通常通常在每个IDR之前周期性地重复SPS和PPS，从而为解码器创建一个随机访问点.这样可以加入已经在进行中的流.
Annex B is commonly used in live and streaming formats such as transport streams, over the air broadcasts, and DVDs. In these formats it is common to repeat the SPS and PPS periodically, usually preceding every IDR thus creating a random access point for the decoder. This enables the ability to join a stream already in progress.
存储H.264流的另一种常见方法是AVCC格式.在这种格式中，每个NALU都以其长度开头(采用大字节序格式).此方法更易于解析，但是您失去了附件B的字节对齐功能.为了使事情复杂，长度可以使用1、2或4个字节进行编码.此值存储在标头对象中.此标头通常称为额外数据"或序列标头".其基本格式如下:
The other common method of storing an H.264 stream is the AVCC format. In this format, each NALU is preceded with its length (in big endian format). This method is easier to parse, but you lose the byte alignment features of Annex B. Just to complicate things, the length may be encoded using 1, 2 or 4 bytes. This value is stored in a header object. This header is often called ‘extradata’ or ‘sequence header’. Its basic format is as follows:
bits
8 version ( always 0x01 )
8 avc profile ( sps[0][1] )
8 avc compatibility ( sps[0][2] )
8 avc level ( sps[0][3] )
6 reserved ( all bits on )
2 NALULengthSizeMinusOne
3 reserved ( all bits on )
5 number of SPS NALUs (usually 1)
repeated once per SPS:
16 SPS size
variable SPS NALU data
8 number of PPS NALUs (usually 1)
repeated once per PPS
16 PPS size
variable PPS NALU data
使用上面的相同示例，AVCC额外数据将如下所示:
Using the same example above, the AVCC extradata will look like this:
0x0000 | 01 64 00 0A FF E1 00 19 67 64 00 0A AC 72 84 44
0x0010 | 26 84 00 00 03 00 04 00 00 03 00 CA 3C 48 96 11
0x0020 | 80 01 00 07 68 E8 43 8F 13 21 30
您会发现SPS和PPS现在已被带外存储.即，与基本流数据分开.这些数据的存储和传输是文件容器的工作，超出了本文档的范围.请注意，即使我们没有使用起始代码，仿真阻止字节仍会插入.
You will notice SPS and PPS is now stored out of band. That is, separate from the elementary stream data. Storage and transmission of this data is the job of the file container, and beyond the scope of this document. Notice that even though we are not using start codes, emulation prevention bytes are still inserted.
此外，还有一个名为NALULengthSizeMinusOne的新变量.这个令人混淆的命名变量告诉我们要使用多少字节来存储每个NALU的长度.因此，如果NALULengthSizeMinusOne设置为0，则每个NALU前面都有一个指示其长度的字节.使用单个字节存储大小，NALU的最大大小为255个字节.那显然很小.对于整个关键帧来说太小了.使用2个字节可使每个NALU达到64k.在我们的示例中可以使用，但仍然是一个很低的限制. 3字节将是完美的，但是由于某些原因，并没有得到普遍支持.因此，到目前为止，最常见的是4个字节，这就是我们在这里使用的:
Additionally, there is a new variable called NALULengthSizeMinusOne. This confusingly named variable tells us how many bytes to use to store the length of each NALU. So, if NALULengthSizeMinusOne is set to 0, then each NALU is preceded with a single byte indicating its length. Using a single byte to store the size, the max size of a NALU is 255 bytes. That is obviously pretty small. Way too small for an entire key frame. Using 2 bytes gives us 64k per NALU. It would work in our example, but is still a pretty low limit. 3 bytes would be perfect, but for some reason is not universally supported. Therefore, 4 bytes is by far the most common, and it is what we used here:
0x0000 | 00 00 02 41 65 88 81 00 05 4E 7F 87 DF 61 A5 8B
0x0010 | 95 EE A4 E9 38 B7 6A 30 6A 71 B9 55 60 0B 76 2E
0x0020 | B5 0E E4 80 59 27 B8 67 A9 63 37 5E 82 20 55 FB
0x0030 | E4 6A E9 37 35 72 E2 22 91 9E 4D FF 60 86 CE 7E
0x0040 | 42 B7 95 CE 2A E1 26 BE 87 73 84 26 BA 16 36 F4
0x0050 | E6 9F 17 DA D8 64 75 54 B1 F3 45 0C 0B 3C 74 B3
0x0060 | 9D BC EB 53 73 87 C3 0E 62 47 48 62 CA 59 EB 86
0x0070 | 3F 3A FA 86 B5 BF A8 6D 06 16 50 82 C4 CE 62 9E
0x0080 | 4E E6 4C C7 30 3E DE A1 0B D8 83 0B B6 B8 28 BC
0x0090 | A9 EB 77 43 FC 7A 17 94 85 21 CA 37 6B 30 95 B5
0x00A0 | 46 77 30 60 B7 12 D6 8C C5 54 85 29 D8 69 A9 6F
0x00B0 | 12 4E 71 DF E3 E2 B1 6B 6B BF 9F FB 2E 57 30 A9
0x00C0 | 69 76 C4 46 A2 DF FA 91 D9 50 74 55 1D 49 04 5A
0x00D0 | 1C D6 86 68 7C B6 61 48 6C 96 E6 12 4C 27 AD BA
0x00E0 | C7 51 99 8E D0 F0 ED 8E F6 65 79 79 A6 12 A1 95
0x00F0 | DB C8 AE E3 B6 35 E6 8D BC 48 A3 7F AF 4A 28 8A
0x0100 | 53 E2 7E 68 08 9F 67 77 98 52 DB 50 84 D6 5E 25
0x0110 | E1 4A 99 58 34 C7 11 D6 43 FF C4 FD 9A 44 16 D1
0x0120 | B2 FB 02 DB A1 89 69 34 C2 32 55 98 F9 9B B2 31
0x0130 | 3F 49 59 0C 06 8C DB A5 B2 9D 7E 12 2F D0 87 94
0x0140 | 44 E4 0A 76 EF 99 2D 91 18 39 50 3B 29 3B F5 2C
0x0150 | 97 73 48 91 83 B0 A6 F3 4B 70 2F 1C 8F 3B 78 23
0x0160 | C6 AA 86 46 43 1D D7 2A 23 5E 2C D9 48 0A F5 F5
0x0170 | 2C D1 FB 3F F0 4B 78 37 E9 45 DD 72 CF 80 35 C3
0x0180 | 95 07 F3 D9 06 E5 4A 58 76 03 6C 81 20 62 45 65
0x0190 | 44 73 BC FE C1 9F 31 E5 DB 89 5C 6B 79 D8 68 90
0x01A0 | D7 26 A8 A1 88 86 81 DC 9A 4F 40 A5 23 C7 DE BE
0x01B0 | 6F 76 AB 79 16 51 21 67 83 2E F3 D6 27 1A 42 C2
0x01C0 | 94 D1 5D 6C DB 4A 7A E2 CB 0B B0 68 0B BE 19 59
0x01D0 | 00 50 FC C0 BD 9D F5 F5 F8 A8 17 19 D6 B3 E9 74
0x01E0 | BA 50 E5 2C 45 7B F9 93 EA 5A F9 A9 30 B1 6F 5B
0x01F0 | 36 24 1E 8D 55 57 F4 CC 67 B2 65 6A A9 36 26 D0
0x0200 | 06 B8 E2 E3 73 8B D1 C0 1C 52 15 CA B5 AC 60 3E
0x0210 | 36 42 F1 2C BD 99 77 AB A8 A9 A4 8E 9C 8B 84 DE
0x0220 | 73 F0 91 29 97 AE DB AF D6 F8 5E 9B 86 B3 B3 03
0x0230 | B3 AC 75 6F A6 11 69 2F 3D 3A CE FA 53 86 60 95
0x0240 | 6C BB C5 4E F3
此格式的一个优点是能够在开始时配置解码器并跳入流的中间.这是一种常见的使用情况，该介质在诸如硬盘驱动器之类的随机访问介质上可用，因此可以在诸如MP4和MKV的常见容器格式中使用.
An advantage to this format is the ability to configure the decoder at the start and jump into the middle of a stream. This is a common use case where the media is available on a random access medium such as a hard drive, and is therefore used in common container formats such as MP4 and MKV.
这篇关于H.264流的序列/图片参数集的可能位置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
