使用 MediaMuxer 混合相机预览 h264 编码的基本流 [英] Muxing camera preview h264 encoded elementary stream with MediaMuxer

问题描述

我正在实施一个关于使用新 MediaCodec 和Android 4.3 的 MediaMuxer API.

I am working on an implementation of one of the Android Test Cases regarding previewTexture recording with the new MediaCodec and MediaMuxer API's of Android 4.3.

通过将 recordingHint 设置为相机参数，我设法以大约 30fps 的帧速率记录预览流.

I've managed to record the preview stream with a framerate of about 30fps by setting the recordingHint to the camera paremeters.

但是，我遇到了延迟/滞后问题，不知道如何解决.当使用相当标准的质量设置(1280x720，比特率约为 8.000.000)录制相机预览时，预览和编码材料偶尔会出现延迟.更具体地说:这种延迟大约每 2-3 秒发生一次，大约需要 300-600 毫秒.

However, I ran into a delay/lag problem and don't really know how to fix that. When recording the camera preview with quite standard quality settings (1280x720, bitrate of ~8.000.000) the preview and the encoded material suffers from occasional lags. To be more specific: This lag occurs about every 2-3 seconds and takes about 300-600ms.

通过跟踪延迟，我发现延迟来自drainEncoder"方法中的以下代码行:

By tracing the delay I was able to figure out the delay comes from the following line of code in the "drainEncoder" method:

mMuxer.writeSampleData(mTrackIndex, encodedData, mBufferInfo);

如果编码器有可用于复用的数据，则循环调用此行.目前我不录制音频，所以只有 h264 流由 MediaMuxer 转换为 mp4 格式.

This line is called in a loop if the encoder has data available for muxing. Currently I don't record audio so only the h264 streams is converted to a mp4 format by the MediaMuxer.

我不知道这是否与延迟有关，但它总是在循环需要两次迭代以使编码器的所有可用数据出列时发生(更具体地说，它总是发生在其中的第一个两次迭代).在大多数情况下，一次迭代就足以使编码器出队.

I don't know if this has something to do with that delay, but it always occurs when the loop needs two iterations to dequeue all available data of the encoder (to be even more specific it occurs always in the first of these two iterations). In most cases one iteration is enough to dequeue the encoder.

由于没有太多关于这些新 API 的在线信息，非常感谢任何帮助！

Since there is not much information online about these new API's any help is very appreciated!

推荐答案

我怀疑你被 MediaMuxer 磁盘写入所困扰.最好的确定方法是在录制期间运行 systrace 并查看暂停期间实际发生的情况.(系统跟踪 docs, 解释，bigflake示例-- 目前只有后者针对 Android 4.3 进行了更新)

I suspect you're getting bitten by the MediaMuxer disk write. The best way to be sure is to run systrace during recording and see what's actually happening during the pause. (systrace docs, explanation, bigflake example -- as of right now only the latter is updated for Android 4.3)

如果是这种情况，您可以通过在单独的线程上运行 MediaMuxer 实例，通过同步队列将 H.264 数据提供给它来缓解问题.

If that's the case, you may be able to mitigate the problem by running the MediaMuxer instance on a separate thread, feeding the H.264 data to it through a synchronized queue.

这些暂停是否定期发生，每 5 秒一次?CameraToMpegTest 示例将编码器配置为每 5 秒输出一个 I 帧(预期帧速率为 30fps)，这导致输出全尺寸帧而不是微小的增量.

Do these pauses happen regularly, every 5 seconds? The CameraToMpegTest example configures the encoder to output an I-frame every 5 seconds (with an expected frame rate of 30fps), which results in a full-sized frame being output rather than tiny deltas.

这篇关于使用 MediaMuxer 混合相机预览 h264 编码的基本流的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！