为什么会出现＆QUOT;不支持的格式＆QUOT;错误，读取H.264连接codeD RTSP流了Android MediaPlayer的？ [英] Why am I getting "Unsupported format" errors, reading H.264 encoded rtsp streams with the Android MediaPlayer?

问题描述

我想告诉H.264 EN codeD RTSP视频在Android设备上。该流从一个树莓派的到来，使用VLC来恩code 的/ dev / VIDEO1 这是一个黑色皮相机局。

I am trying to show H.264 encoded rtsp video on an Android device. The stream is coming from a Raspberry Pi, using vlc to encode /dev/video1 which is a "Pi NoIR Camera Board".

vlc-wrapper -vvv v4l2:///dev/video1 --v4l2-width $WIDTH --v4l2-height $HEIGHT --v4l2-fps ${FPS}.0 --v4l2-chroma h264 --no-audio --no-osd --sout "#rtp{sdp=rtsp://:8000/pi.sdp}" :demux=h264 > /tmp/vlc-wrapper.log 2>&1

我用很小的Andr​​oid code现在：

I am using very minimal Android code right now:

final MediaPlayer mediaPlayer = new MediaPlayer();
mediaPlayer.setDisplay(holder);
try {
  mediaPlayer.setDataSource(url);
  mediaPlayer.prepare();

和得到一个prepare失败：状态=为0x1 IOException异常。当我看到日志，我看到这样

and getting a "Prepare failed.: status=0x1" IOException. When I look at the logs, I see lines like

06-02 16:28:05.566 W/APacketSource(  316): Format:video 0 RTP/AVP 96  / MIME-Type:H264/90000
06-02 16:28:05.566 W/MyHandler(  316): Unsupported format. Ignoring track #1.
06-02 16:28:05.566 I/MyHandler(  316): SETUP(1) completed with result -1010 (Unknown error 1010)

这是一个系统的过程中来了。 Grepping这些消息指向 libstagefright / RTSP 来源和似乎的意思是， ASessionDescription :: getDimensions 在 APacketSource :: APacketSource通话构造失败。这似乎并不像它应该发生，因为VLC肯定知道什么尺寸的输出：

coming from a system process. Grepping for these messages points to the libstagefright/rtsp sources, and seems to mean that the ASessionDescription::getDimensions call in the APacketSource::APacketSource constructor is failing. This doesn't seem like it should be happening, because VLC certainly knows what dimensions to output:

[0x1c993a8] v4l2 demux debug: trying specified size 800x600
[0x1c993a8] v4l2 demux debug: Driver requires at most 262144 bytes to store a complete image
[0x1c993a8] v4l2 demux debug: Interlacing setting: progressive
[0x1c993a8] v4l2 demux debug: added new video es h264 800x600

什么的似乎的要发生的是， ASessionDescription :: getDimensions 正在寻找框架尺寸的属性（貌似合式） DESCRIBE 结果

What seems to be happening is that ASessionDescription::getDimensions is looking for a framesize attribute in the (seemingly well-formed) DESCRIBE results

06-02 16:28:05.566 I/MyHandler(  316): DESCRIBE completed with result 0 (Success)
06-02 16:28:05.566 I/ASessionDescription(  316): v=0
06-02 16:28:05.566 I/ASessionDescription(  316): o=- 15508012299902503225 15508012299902503225 IN IP4 pimple
06-02 16:28:05.566 I/ASessionDescription(  316): s=Unnamed
06-02 16:28:05.566 I/ASessionDescription(  316): i=N/A
06-02 16:28:05.566 I/ASessionDescription(  316): c=IN IP4 0.0.0.0
06-02 16:28:05.566 I/ASessionDescription(  316): t=0 0
06-02 16:28:05.566 I/ASessionDescription(  316): a=tool:vlc 2.0.3
06-02 16:28:05.566 I/ASessionDescription(  316): a=recvonly
06-02 16:28:05.566 I/ASessionDescription(  316): a=type:broadcast
06-02 16:28:05.566 I/ASessionDescription(  316): a=charset:UTF-8
06-02 16:28:05.566 I/ASessionDescription(  316): a=control:rtsp://192.168.1.35:8000/pi.sdp
06-02 16:28:05.566 I/ASessionDescription(  316): m=video 0 RTP/AVP 96
06-02 16:28:05.566 I/ASessionDescription(  316): b=RR:0
06-02 16:28:05.566 I/ASessionDescription(  316): a=rtpmap:96 H264/90000

本的看起来的它可能是一个Stagefright错误：它知道（或者应当知道），它有一个H.264连接codeD流，但它似乎是期待H. 263 框架尺寸属性。因此，我的问题：

This looks like it may be a Stagefright bug: It knows (or should know) that it has a H.264 encoded stream, yet it seems to be expecting a H.263 framesize attribute. Hence my questions:

1. 难道我读的来源吗？在 ASessionDescription :: getDimensions问题打电话？ （是否stagefright实际上只支持H.263流？）
2. 或者是丕方code错误以某种方式？
3. 还是我只是缺少在我的客户端code的关键一步还是两个？

1. Am I reading the sources right? Is the problem in the ASessionDescription::getDimensions call? (Does stagefright only actually support H.263 streaming?)
2. Or is the Pi-side code wrong in some way?
3. Or am I just missing a key step or two in my client-side code?

在的MediaPlayer 文档说-1010是的 MEDIA_ERROR_UNSUPPORTED ：档案是符合相关的编码标准或文件规范，但媒体框架不支持的功能这使我怀疑问题出在标准渐进式下载的问题。也就是说，支持的媒体格式说：

Update, 20140606:

The MediaPlayer docs say that -1010 is MEDIA_ERROR_UNSUPPORTED: "Bitstream is conforming to the related coding standard or file spec, but the media framework does not support the feature." This makes me wonder if the problem is the 'standard' progressive download issue. That is, Supported Media Formats says

对于来流式传输视频内容的HTTP或RTSP [中] MPEG-4容器]的 MOOV 原子必须precede任何 MDAT 原子，但一定要成功的 ftyp 原子

For video content that is streamed over HTTP or RTSP [in a] MPEG-4 [container] the moov atom must precede any mdat atoms, but must succeed the ftyp atom

而大多数流把 MOOV 原子最后。

while most streams put the moov atom last.

我并不知道如何验证这一点，但！

I am not at all sure how to verify this, though!

• 在我没有看到 MOOV 或 ftyp 原子的VLC源。 （我的告诉的是VLC只是流，在这里;实际H264内容是走出了摄像头驱动）
• 在我没有看到 MOOV 或 ftyp 原子中的 https://github.com/raspberrypi Linux或用户级分支机构。 （也许我只是grepping为错误的东西，虽然。）
• 当我有VLC保存流，我与 MOOV 在 MDAT 但当然，MP4文件VLC可以做一些代码转换，在这里。

• I see no moov or ftyp atoms in the vlc source. (I am told that vlc is just streaming, here; that the actual H264 content is coming out of the camera driver.)
• I see no moov or ftyp atoms in the https://github.com/raspberrypi linux or userland branches. (Maybe I'm just grepping for the wrong things, though.)
• When I have vlc save the stream, I get an mp4 file with moov before mdat, but of course vlc could be doing some transcoding, here.

借助 GPACOsmo4球员能在Android 4.3的平板显示流。严重（比VLC较为滞后的一台笔记本电脑，而且容易死机），但它的可以显示出来。

The GPAC "Osmo4" player can display the stream on an Android 4.3 tablet. Badly (more lag than VLC on a laptop, and prone to lockups) but it can display it.

当我再次尝试grepping的VLC源（不区分大小写，无字的方向，这个时候）我没有找到FOURCC宏定义 MOOV 和 ftyp 原子模块/复用/ mp4.c ，从而迅速导致了 - SOUT-MP4播放器-faststart （和 - 没有SOUT-MP4-的fastStart ）。开关......不作任何区别

When I tried grepping the VLC sources again (case-insensitive and without word-orientation, this time) I did find the FOURCC macros defining the moov and ftyp atoms in modules/mux/mp4.c, which quickly led to the --sout-mp4-faststart (and --no-sout-mp4-faststart) switches ... which don't make any difference.

所以，它看起来像它实际上可能不是一个原子排序问题。这是很好的了解，如果它关闭掉一整类死角的，但它确实给我留下撞我的头靠在墙上（这似乎总是做更多的伤害到我的头上，而不是墙）没有线索。

So, it looks like it may actually not be an atom-ordering issue. That's good to know, if it closes off a whole class of dead-ends, but it does leave me banging my head against the wall (which always seem to do more damage to my head than to the wall) without a clue.

我编译VLC对于Android，它可以显示VLC的PI产生的数据流。它把在屏幕的左上角的图像;我试着写我自己的皮肤他们。所以，并不能找到任何旋钮，将让我放大到表面或什么的。 （加上.apk文件来约12M！）

I compiled VLC for Android, and it can display the stream generated by VLC on the pi. It puts the image in the top-left of the screen; I tried writing my own skin for their .so, and couldn't find any 'knobs' that would let me zoom-to-surface or whatever. (Plus the .apk came to about 12M!)

于是，我找到了相关的RFC和写我自己的RTSP客户端。或者尝试：我可以解析 SDP ，并产生足够的有效的 RTSP 获得 RTP 并RTCP数据报了，我可以分析RTP和RTCP头。但是，即使在SDP声称提供M = 0的视频RTP / AVP 96和= rtpmap： 96 H264 / 90000，该媒体codeC 将不会在我的面上显示视频，不管它在我的平板电脑三H264 codeCS我传递给<一href="http://developer.android.com/reference/android/media/Media$c$cc.html#createBy$c$ccName(java.lang.String)"相对=nofollow>媒体codec.createBy codecName（），当我看RTP有效载荷，我不是太惊讶：我没有看到的NAL同步模式的任何数据包的任何地方。

So, I found the relevant RFCs and wrote my own RTSP client. Or tried to: I can parse the SDP and generate enough valid RTSP to get RTP and RTCP datagrams, and I can parse the RTP and RTCP headers. But even though the SDP claims to deliver m=video 0 RTP/AVP 96 and a=rtpmap:96 H264/90000, the MediaCodec won't display video on my surface, no matter which of the three H264 codecs on my tablet I pass to MediaCodec.createByCodecName(), and when I look at the RTP payloads, I'm not too surprised: I don't see the NAL sync pattern anywhere in any of the packets.

相反，他们的所有的开始，无论是 21 9A __ 22 FF （通常情况下）或者偶尔 3C 81 9A __ 22 FF ，其中__似乎永远是一个甚至数上升了2每个数据包。我不认可这种模式 - ？你

Instead, they all start with either 21 9A __ 22 FF (usually) or occasionally 3C 81 9A __ 22 FF, where the __ seems to always be an even number that goes up by 2 each packet. I don't recognize this pattern - do you?

原来，H264分组不具有开始与NAL同步模式 - 这是只有必要其中NAL单元可以被嵌入在一个较大的数据流。我的RTP包在 RFC 6184 格式。

Turns out that H264 packets don't have to start with the NAL sync pattern - that's only necessary where NAL Units may be embedded in a larger data stream. My RTP packets are in RFC 6184 format.

推荐答案

中拥有大量的死胡同后，我可以显示在Android一个H264的RTSP流 SurfaceView 。这个答案是唯一的排序的答案，因为我现在还不能满足我原来的三个问题，但即使是完全错误和快捷方式，因为它是，我的75K的apk是很多比VLC更好地为Android或在osmo4播放器：具有亚秒级时延（！至少当发送者和接收者是相同的无线路由器），并填补了 SurfaceView

After an amazing amount of dead-ends, I can show a H264 RTSP stream on an Android SurfaceView. This answer is only sort of an answer because I still can't address my original three questions, but even full of bug and shortcuts as it is, my 75K apk is a lot better than Vlc for Android or the osmo4 player: It has sub-second latency (at least when the sender and the receiver are on the same wifi router!) and fills the SurfaceView.

几个外卖，帮助任何人试图做类似的话：

A few takeaways, to help anyone trying to do anything similar:

• 在所有的输入缓冲区传递给<一个href="http://developer.android.com/reference/android/media/Media$c$cc.html#queueInputBuffer(int,%20int,%20int,%20long,%20int)vvvv"相对=nofollow> 媒体codec.queueInputBuffer（） 必须以00 00 01的同步模式。
• 您可以<一href="http://developer.android.com/reference/android/media/Media$c$cc.html#configure(android.media.MediaFormat,%20android.view.Surface,%20android.media.MediaCrypto,%20int)"相对=nofollow> 配置（） 和的 启动（） 中的codeC马上 - 但不排队任何正常输入缓冲器，直到你看到两者的SPS（NALU code 7）和PPS（NALU code 8）数据包。 （这可能不是0x67和0x68 - 在nal_ref_idc值位应该为非零值，但不一定会11 FWIW， VLC 似乎总是给我01）
• 通过SPS / PPS的包的几乎的正常 - 通过的 BUFFER_FLAG_ codeC_CONFIG 标志 queueInputBuffer（）。尤其是，不要试图把它们放在一个CSD-0缓冲连接到 MediaFormat ！
• 当你看到（一）未接框架（S）（即的，你看到的RTP序列号跳转）的不要拨打的 codec.flush（） 的！刚刚跳过部分帧，不排队一个缓冲溶液，直到下一次全画幅。

• All input buffers you pass to MediaCodec.queueInputBuffer() must start with the 00 00 01 sync pattern.
• You can configure() and start() the codec right away - but don't queue any 'normal' input buffers until you've see both an SPS (NALU code 7) and PPS (NALU code 8) packet. (These might not be 0x67 and 0x68 - the "nal_ref_idc" bits should be non-zero but will not necessarily be 11. Fwiw, vlc seems to always give me 01.)
• Pass the SPS/PPS packets almost normally - pass the BUFFER_FLAG_CODEC_CONFIG flag to queueInputBuffer(). In particular, don't try to put them in a "csd-0" buffer attached to the MediaFormat!
• When you see (a) missed frame(s) (i.e. you see a jump in RTP sequence number) do not call codec.flush()! Just skip the partial frame, and don't queue up a bufer until the next full frame.

这篇关于为什么会出现＆QUOT;不支持的格式＆QUOT;错误，读取H.264连接codeD RTSP流了Android MediaPlayer的？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！