如何在Android exoplayer中播放原始NAL单元? [英] How to play raw NAL units in Android exoplayer?
问题描述
我知道exoplayer支持RTSP，但是我需要适用于许多操作系统的播放器的C ++代码，因此在传递给exoplayer之前，我需要先将C ++中的RTP数据包解析为NAL单元
我找到了一种方法，可以使用live555 解码RTP数据包并提取其NAL单位.根据 ExoPlayer的文档:
I found a way to decode RTP packets using live555 and extract its NAL units. According to ExoPlayer's documentation:
所有ExoPlayer实现共有的组件是:
Components common to all ExoPlayer implementations are:
MediaSource，它定义要播放的媒体，加载媒体，并从中读取加载的媒体.
A MediaSource that defines the media to be played, loads the media, and from which the loaded media can be read.
MediaSource是 在播放开始时通过ExoPlayer.prepare注入. ...
A MediaSource is injected via ExoPlayer.prepare at the start of playback. ...
所以我需要一个自定义的MediaSource，可以从我的C ++代码中提取NAL单元.
So I need a custom MediaSource that can extract NAL units from my C++ code.
在 MediaSource的类参考中我们可以看到已经有一些MediaSource可用.我也许 SmoothStreaming MediaSource 可以工作，但是没有确切说明它的功能，在其构造函数中，我必须提供Uri或SsManifest(内容).
At the class reference for MediaSource we can see that there are already some MediaSources available. I though maybe SmoothStreaming MediaSource could work but there's no description of what it does exactly and in its constructor I have to provide an Uri or an SsManifest (what).
我可以看到存在一个该库中的NAL单位实用程序，所以也许事情已经完成了一半
I can see that there exists a NAL unit utility in this library so maybe things are already half done
那么，如何构建或使用已经可用的MediaSource来读取供ExoPlayer播放的NAL单元?
So how to build or use an already available MediaSource to read NAL units for ExoPlayer to play?
作为补充，您如何将NAL单元从C ++传递给Java?在我发现的代码中，它只是将它们存储在C ++缓冲区中.我应该以某种方式在Java中读取此缓冲区吗?
As an additinal, how would you pass the NAL units from C++ to Java? In the code I found it's simply storing them in a C++ buffer. Should I read this buffer in Java somehow?
更新:
我一直在研究该库的工作方式.这一切都始于 MediaSource 具有诸如 Extractor 和数据源.似乎 ExtractorMediaSource 是MediaSource，您可以在其中提供自己的Extractor和DataSource.
I've been researching how this library works. It all begins with a MediaSource which have objects like Extractor and DataSource. Seems that ExtractorMediaSource is a MediaSource where you can provide your own Extractor and DataSource.
据我了解，DataSource是一个类，它从任何可能的位置获取原始字节，无论是文件读取还是网络数据包.根据库中可用的Extractor类(例如Mp4Extractor和Mp3Extractor)，Extractor可以解释从DataSource读取的数据. Extractor界面中的两种主要方法是:
As I understood, DataSource is a class that gets the raw bytes from any possible place, be it a file read or a network packet. Based on the available Extractor classes on the library like Mp4Extractor and Mp3Extractor, an Extractor is something that will interpret the data read from DataSource. The two main methods from the Extractor interface are:
void init(ExtractorOutput output)
int read(ExtractorInput input, PositionHolder seekPosition)
我不知道ExtractorInput和ExtractorInput的用途是什么，但是它们看起来很重要.
I don't know what are ExtractorInput and ExtractorInput for, but they look important.
因此Extractor以某种方式从DataSource中读取内容，对其进行解析并将其发送到
So somehow Extractor reads from DataSource, parses it and sends to Renderer in a common format?
我需要知道这种通用格式是什么，以便可以解析从自定义DataSource中读取的NAL单元.
I need to know how is this common format so I can parse the NAL units that I read from a custom DataSource.
推荐答案
您无法使用ExoPlayer播放NAL单元或原始H.264流.
图片数据必须存在于 受支持的容器/格式.
You cannot play NAL units or raw H.264 stream with ExoPlayer.
The picture data must exist within a supported container/format.
尚不清楚您神秘的C ++代码在做什么，是否是 strong> NDK 设置?它在Android解码中起什么作用?您是说无法将 [来自C ++函数] 的数据数组传递给某些Android函数作为函数参数吗?就像这样 Java to C ++ 设置?不清楚您真正的问题是什么...
It's not clear what your mysterious C++ code is doing, is it an NDK setup? What's its role in Android decoding? Are you saying you're unable to pass [from the C++ function] a data array into some Android function as function parameter? Is it something like this Java to C++ setup? It's not clear what your real problem is...
如果您坚持使用Exoplayer，我可以告诉您FLV是(在容器列表上)可能是最好的选择，因为它可以实时构建(重新混合).首先，您创建一个FLV标头，其中包含SPS和PPS数据，然后是关键帧(从第一个NAL中提取的H264数据).您必须熟悉FLV字节结构，但是每个帧头大约为13个字节，后跟NAL数据，对每个帧重复直到结束.这将是实时转码.
If you insist on Exoplayer, I can tell you that FLV is the one (on containers list) that might be the best option since it can be built in real-time (re-muxing). You first create an FLV header that holds SPS and PPS data then followed by keyframe (extracted H264 data from the first NAL). You'll have to get familiar with FLV bytes structure, but each frame header is around 13 bytes followed by NAL data, repeat for each frame until end. This woud be realtime transcoding.
作为第二个选项，对于Android，您可以只使用
As a second option, for Android, you could just use MediaCodec to decode the H264 as extracted from the NAL units. Here is a useful example source. Just use as:
MediaCodec.createDecoderByType("video/avc"); //then later give NAL units
Study also functions of this other source for ideas of how it works to via Android's own decoder.
其他起点:
Decode H264 raw stream using mediacodec
这篇关于如何在Android exoplayer中播放原始NAL单元?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
