Android:同时录制和流式传输 [英] Android: Recording and Streaming at the same time

问题描述

这并不是一个真正的问题，而是展示了我为解决我所面临的最具挑战性的功能之一所做的所有尝试.

This is not really a question as much as it is a presentation of all my attempts to solve one of the most challenging functionalities I was faced with.

我使用 libstreaming 库将实时视频流式传输到 Wowza Server 并且我需要同时将其记录在 SD 卡中.为了从社区收集新的想法，我在下面展示了我的所有尝试.

I use libstreaming library to stream realtime videos to Wowza Server and I need to record it at the same time inside the SD card. I am presenting below all my attempts in order to collect new ideias from the community.

开发

我们在 libstreaming 库中创建了一个拦截器，将所有发送的字节复制到一个 mp4 文件中.Libstreaming 通过 LocalSocket 将字节发送到 Wowza 服务器.它使用 MediaRecorder 访问设备的摄像头和麦克风，并将输出文件设置为 LocalSocket 的输入流.我们所做的是围绕这个从 InputStream 扩展的输入流创建一个包装器，并在其中创建一个 File 输出流.因此，每次 libstreaming 对 LocaSocket 的输入流执行读取时，我们都会将所有数据复制到输出流中，尝试创建一个有效的 MP4 文件.

We created an interception in libstreaming library to copy all the sent bytes to a mp4 file. Libstreaming sends the bytes to Wowza server through a LocalSocket. It users MediaRecorder to access the camera and the mic of the device and sets the output file as the LocalSocket's input stream. What we do is create a wrapper around this input stream extending from InputStream and create a File output stream inside it. So, every time libstreaming executes a reading over the LocaSocket's input stream, we copy all the data to the output stream, trying to create a valid MP4 file.

障碍

当我们尝试读取文件时，它已损坏.我们意识到 MP4 文件中缺少元信息.特别是 moov 原子.我们试图延迟流媒体的关闭，以便有时间发送这个标头(这仍然是一个猜测)，但它没有奏效.为了测试这些数据的一致性，我们使用付费软件来尝试恢复视频，包括标题.它变得可玩，但主要是绿屏.所以这变成了一个不可信的解决方案.我们还尝试使用untrunc"，一个免费的开源命令行程序，它甚至无法启动恢复，因为没有 moov atom.

When we tried to read the file, it is corrupted. We realized that there are meta information missing from the MP4 file. Specifically the moov atom. We tried to delay the closing of the streaming in order to give time to send this header (this was still a guessing) but it didn't work. To test the coherence of this data, we used a paid software to try to recover the video, including the header. It became playable, but it was mostly green screen. So this became an not trustable solution. We also tried using "untrunc", a free open source command line program and it couldn't even start the recovery, since there was no moov atom.

开发

FFMPEG 有一个带有 Java 接口的 gradle 插件，可以在 Android 应用程序中使用它.我们认为我们可以通过命令行访问摄像机(它可能在/dev/video0"中)并将其发送到媒体服务器.

FFMPEG has a gradle plugin with a java interface to use it inside Android apps. We thought we could access the camera via command line (it is probably in "/dev/video0") and sent it to the media server.

障碍

我们在尝试访问相机时收到错误权限被拒绝".解决方法是将设备设为 root 以访问它，但这会使手机失去保修并可能使它们变砖.

We got the error "Permission Denied" when trying to access the camera. The workaround would be to root the device to have access to it, but it make the phones loose their warranty and could brick them.

开发

我们尝试通过 MediaRecorder 使 FFMPEG 流成为手机内录制的 mp4 文件

We tried to make FFMPEG stream a mp4 file being recorded inside the phone via MediaRecorder

障碍

FFMPEG 无法流式传输尚未完成录制的 MP4 文件.

FFMPEG can not stream MP4 files that are not yet done with the recording.

开发

Libstreaming 使用LocalServerSocket 作为app 和服务器的连接，所以我们认为我们可以使用连接LocalServerSocket 本地地址的ffmpeg 将流直接复制到SD 卡内的本地文件中.流开始后，我们还运行了 ffmpeg 命令开始将数据记录到文件中.使用ffmpeg，我们相信它会以正确的方式创建一个MP4文件，这意味着包含moov atom header.

Libstreaming uses LocalServerSocket as the connection between the app and the server, so we thought that we could use ffmpeg connected with LocalServerSocket local address to copy the streaming directly to a local file inside the SD card. Right after the streaming started, we also ran the ffmpeg command to start recording the data to a file. Using ffmpeg, we believed that it would create a MP4 file in the proper way, which means with the moov atom header included.

障碍

创建的地址"无法通过命令行读取，作为手机内部的本地地址.所以复制是不可能的.

The "address" created is not readable via command line, as a local address inside the phone. So the copy is not possible.

开发

OpenCV 是一个开源的跨平台库，可为计算机视觉实验和应用程序提供构建块.它提供了用于捕获、处理和呈现图像数据的高级接口.它有自己的 API 来连接设备摄像头，所以我们开始研究它，看看它是否具有同时传输和记录的必要功能.

OpenCV is an open-source, cross-platform library that provides building blocks for computer vision experiments and applications. It offers high-level interfaces for capturing, processing, and presenting image data. It has their own APIs to connect with the device camera so we started studding it to see if it had the necessary functionalities to stream and record at the same time.

障碍

我们发现该库并没有真正定义为执行此操作，而是更多地定义为图像数学处理.我们甚至得到了使用 libstreaming 的建议(我们已经这样做了).

We found out that the library is not really defined to do this, but more as image mathematical manipulation. We got even the recommendation to use libstreaming (which we do already).

开发

Kickflip 是一种媒体流服务，提供自己的 SDK，用于在 android 和 IOS 中进行开发.它还使用 HLS 而不是 RTMP，后者是一种较新的协议.

Kickflip is a media streaming service that provides their own SDK for development in android and IOS. It also uses HLS instead of RTMP, which is a newer protocol.

障碍

他们的 SDK 要求我们创建一个带有摄像头视图的 Activity，该视图占据设备的整个屏幕，从而破坏了我们应用的可用性.

Their SDK requires that we create a Activity with camera view that occupies the entire screen of the device, breaking the usability of our app.

开发

我们开始咨询 Play 商店中已有应用程序的其他开发者，这些应用程序已经传输到服务器.

We started consulting other developers of app's already available in the Play Store, that stream to servers already.

障碍

与这些开发人员取得联系后，他们保证无法使用此技术同时进行录制和流式传输.更重要的是，我们必须使用 Adob​​e Air 从头开始​​重做整个应用程序.

Getting in touch with those developers, they reassured that would not be possible to record and stream at the same time using this technology. What's more, we would have to redo the entire app from scratch using Adobe Air.

开发

在这个伟大的项目之后，我们开始使用 WebRTC.我们在 NODEJS 服务器中包含了信令服务器，并开始通过套接字进行标准握手.我们仍在通过 webrtc 在本地录制和流媒体之间切换.

We started using WebRTC following this great project. We included the signaling server in our NODEJS server and started doing the standard handshake via socket. We were still toggling between local recording and streaming via webrtc.

障碍

Webrtc 不适用于所有网络配置.除此之外，相机获取都是本地代码，这使得尝试复制字节或拦截它变得更加困难.

Webrtc does not work in every network configuration. Other than that, the camera acquirement is all native code, which makes a lot harder to try to copy the bytes or intercept it.

推荐答案

如果你愿意放弃 libstreaming，有一个库可以轻松地同时流式传输和录制到本地文件.

If you are willing to part with libstreaming, there is a library which can easily stream and record to a local file at the same time.

https://github.com/pedroSG94/rtmp-rtsp-stream-client-java

克隆项目并运行示例应用程序.例如，点击默认 RTSP".输入您的端点.点击开始流"，然后点击开始录制".然后点击停止流"和停止记录".我已经用 Wowza Server 测试过它，它运行良好.该项目还可以用作库而不是独立的应用程序.

Clone the project and run the sample app. For example, tap "Default RTSP." Type in your endpoint. Tap "Start stream" then tap "Start record." Then tap "Stop Stream" and "Stop record." I've tested this with Wowza Server and it works well. The project can also be used as a library rather than standalone app.

这篇关于Android:同时录制和流式传输的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！