如何将H.264 UDP数据包转换为可播放的媒体流或文件(碎片整理) [英] How to convert H.264 UDP packets to playable media stream or file (defragmentation)

问题描述

在将SDP会话的UDP流转换为可解码的H.264流时，我缺少一些基本的东西.我正在使用支持H.264的相机进行测试，并且可以直接通过播放器播放视频流.当我尝试播放转换后的流时，播放器将无法识别该流(缺少标头错误).但是，我必须对UDP流进行解码才能将其集成到Java应用程序中，该应用程序周围有一些解码器.

I am missing some fundamental thing in translating an UDP stream of a SDP session into a decodable H.264 stream. I am testing with a H.264 capable camera and can play the stream with a player directly. When I try to play the translated stream it will not be recognized by the player (missing header error). However I have to decode the UDP stream to be able to integrate this in a Java application for which there are some decoders around.

对于以下问题，我已经看到了很好的答案:

I have seen very good answers to following questions already:

1. 使用ffmpeg在RTP上解码H264视频的问题(libavcodec)

1. How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter
2. Problem to Decode H264 video over RTP with ffmpeg (libavcodec)

两者之间都有一些细微的差异，令人困惑(请参阅下文).

Both have some small differences which are confusing (see below).

但是首先让我们看一下简单的部分.正如我从摄像机看到的那样，发送了SPS和PPS数据包.所有其余的数据包都是碎片索引帧或没有索引的帧.

But first let us look at the easy part. As I see the from the camera there are SPS and PPS packets sent. All the remaining packets are fragmented frames indexed or not.

对于所有没有帧的数据包(在我的情况下仅是NALUnitType 7和8)，我剥离了RTP标头(12字节)，并在开头添加了3 x 0字节和1 x 1的起始字节(00 00 00 01).

For all the packets without frames (only NALUnitType 7 and 8 in my case) I strip of the RTP Header (12 Bytes) and add starting bytes 3 x 0 bytes and 1 x 1 in front (00 00 00 01).

对于所有分段的帧数据包，我根据答案1的描述对其进行了重构.因此，详细而言，这意味着: RTP标头的条带(只需将其用于数据验证). 然后从有效载荷中解码片段信息:

For all fragmented frame packets I reconstruct them according the description of the answer 1. So in detail this means: Strip of the RTP header (just use this for data verification). Then decode from the payload the fragment information:

第一个字节:[3 NAL UNIT BITS | 5片段类型位]
第二个字节:[起始位|结束位|保留位| 5 NAL UNIT BITS]

First byte: [ 3 NAL UNIT BITS | 5 FRAGMENT TYPE BITS]
Second byte: [ START BIT | END BIT | RESERVED BIT | 5 NAL UNIT BITS]

如果设置了起始位，则会有一个新的有效负载头构造如下:[3 NAL UNIT BITS(从第一个字节开始)| 5个NAL UNIT BITS(从第二个字节开始)]
对于非idr切片，这给我们提供了NALUnitType 1；对于idr切片，这给了我们NALUnitType 5.符合规范.

If start bit is set there is a new payload header constructed as this: [3 NAL UNIT BITS (from first byte)| 5 NAL UNIT BITS (from second byte)]
This gives us a NALUnitType 1 for an non idr slice or a 5 for an idr slice. Which is according to the specification.

我采用了这个新的有效载荷标头(1个字节)，并将没有2个字节标头的有效载荷附加到新的程序包中.以相同的方式添加所有连续的片段(12个字节的RTP标头带，2个字节的条带类型的单元类型信息)，直到看到结束位信息为止.看到结尾时，我将开始字节(00 00 00 01)放在此数据包的前面，并将其写出到流中.

I take this new payload header (1 byte) and attach the payload without the 2 bytes header into a new package. All consecutive fragments are added the same way (strip of 12 bytes RTP header, strip of 2 bytes of unit type information) until there is an end bit information seen. When the end is seen I put start bytes (00 00 00 01) in front of this packet and write it out to the stream.

问题是由于未知原因无法解码.我已阅读的答案中答案2的不同之处在于，有效载荷头的第二个字节也可能会放入转换后的数据包中.但是我都尝试了两次，仍然没有运气.

The problem is it can not be decoded for unknown reason. The difference in answer 2 of the answers I have read is that the second byte of the payload header might be put into the translated packet as well. But I tried both and still no luck.

新构建的流中可能还缺少其他内容吗?还是我在整理碎片时弄错了?

Probably there is something other missing in the newly constructed stream ? Or do I make a mistake in the defragmentation?

推荐答案

托马斯(Thomas)

Thomas,

我正试图自己理解所有这些.在我看来，通过阅读以下内容:

I'm trying to understand all of this myself. It looks to me, from reading this: How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter that your "start bytes" is off by one byte. I think it's 3 bytes, not four... as in: 00 00 01

也许是麻烦所在.

这篇关于如何将H.264 UDP数据包转换为可播放的媒体流或文件(碎片整理)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！