在目标C中为现场音频和视频广播创建rtsp客户端 [英] Creating a rtsp client for live audio and video broadcasting in objective C

问题描述

我正在尝试创建一个直播音频和视频的RTSP客户端。我在链接 http://www.gdcl.co.uk/downloads.htm <修改了iOS代码/ a>并能够正确地将视频广播到服务器。但现在我面临广播音频部分的问题。在链接示例中，代码的编写方式是将视频数据写入文件，然后从文件中读取数据并将NALU的视频数据包上传到RTSP服务器。

I am trying to create a RTSP client which live broadcast Audio and Video. I modified the iOS code at link http://www.gdcl.co.uk/downloads.htm and able to broadcast the Video to server properly. But now i am facing issues in broadcasting the audio part. In the link example the code is written in such a way that it writes the Video data to file and than reads the data from the file and upload the NALU's video packets to RTSP server.

对于音频部分，我不知道如何继续。现在我试过的是从麦克风获取音频缓冲区，然后通过添加RTP标头和ALU直接将它广播到服务器..但是这种方法不能正常工作，因为音频开始滞后，滞后随着时间的推移而增加。如果有更好的方法来实现这一目标并使用唇形音频/视频，有人可以告诉我。

For Audio part i am not sure how to proceed on it. Right now what i have tried is that get the audio buffer from mic and than broadcast it to the server directly by adding RTP headers and ALU.. but This approach is not properly working as Audio starts lagging behind and lag increases with time. Can someone let me know if there is some better approach to achieve this and with lip sycn audio/video.

推荐答案

您是否丢失了客户端上的任何数据包？如果是这样，你需要留下空间。如果您收到数据包1,2,3,4,6,7，您需要为丢失的数据包留出空间（5）。

Are you losing any packets on the client? If so, you need to leave "space." If you receive packet 1,2,3,4,6,7, You need to leave space for the missing packet (5).

另一种可能性是什么被称为时钟漂移问题。客户端和服务器上的时钟（晶体）彼此不完全同步。

这可能是由环境，温度变化等引起的。

The other possibility is a what is known as a clock drift problem. The clock (crystal) on your client and server are not perfectly in sync with each other.
This can be caused by environment, temperature changes, etc.

让我们说在一个完美的世界中，您的服务器正在以48000赫兹生成音频样本20毫秒音频样本。您的客户正在使用48000赫兹的采样率播放它们。实际上，您的客户端和服务器并不完全是48000hz。您的服务器可能是48000.001，您的客户端可能是47999.9998。因此，您的服务器可能比您的客户端提供更快的速度，反之亦然。您要么过快地使用数据包，要么在运行缓冲区或滞后太远并且溢出客户端缓冲区。在你的情况下，听起来客户端播放速度太慢而且慢慢落后于服务器。你可能每分钟只会落后几毫秒，但这个问题会继续存在，看起来就像一部20世纪70年代的嘴唇同步功夫电影。

Let's say in a perfect world your server is producing audio samples 20ms audio samples at 48000 hz. Your client is playing them back using a sample rate of 48000 hz. Realistically your client and server are not exactly 48000hz. Your server might be 48000.001 and your client might be 47999.9998. So your server might be delivering faster than your client or vise versa. You would either consume packets too fast and under run the buffer or lag too far behind and overflow the client buffer. In your case, it sounds like the client is playing back too slow and slowly lagging behind the server. You might only lag a couple milliseconds per minute but the issue will keep continuing and it will look like a 1970s lip synced Kung Fu movie.

在其他设备中，通常会有一个通用的时钟线，以保持同步。例如，摄像机时钟，midi时钟。多轨记录器时钟。

In other devices, there is often a common clock line to keep things in sync. For example, Video camera clocks, midi clocks. multitrack recorder clocks.

当您通过IP传送数据时，客户端和服务器之间不共享公共时钟。因此，您的问题涉及在不同设备之间同步时钟。我已经使用这种通用方法成功地解决了这个问题：

When you deliver data over IP, there is no common clock shared between a client and server. So your issue concerns syncing clocks between disparate devices with no. I have successfully solved this problem using this general approach:

• A）让客户端计算一段时间内进来的数据包的速率时间。


• B）让客户端计算消耗（播放）数据包的速率。


• C）调整采样率基于A和B的客户。

因此，您的客户要求您调整播放的采样率。所以是的，你玩得更快或更慢。请注意，播放速率的变化将非常微妙。您可以将采样率设置为48000.0001 hz而不是48000 hz。音调的差异是人类无法察觉的，因为它只会导致音高差异百分之几。我解释了一种非常简化的方法。在开发这样的控制系统时，必须考虑许多其他细微差别和边缘情况。你不只是设置并忘记它。您需要一个控制系统来管理播放。

So your client requires that you adjust the sample rate of the playback. So yes you play it faster or slower. Note that the playback rate change will be very very subtle. You might set the sample rate to be 48000.0001 hz instead of 48000 hz. The difference in pitch would be undetectable by humans as it would only cause a fraction a cent difference in pitch. I gave an explanation of a very simplified approach. There many other nuances and edge cases that must be considered when developing such a control system. You don't just set it and forget it. You need a control system to manage the playback.

一个有趣的测试来证明这一点是将两个设备放在完全相同的文件中。长录音（比如3小时）是最好的。同时启动它们。播放3小时后，您会发现其中一个先于另一个。

An interesting test to demonstrate this is to take two devices with the exact same file. A long recording (say 3 hours) is best. Start them at the same time. After 3 hours of playback, you will notice that one is ahead of the other.

这篇文章解释了流媒体音频和视频不是一项简单的任务。

这篇关于在目标C中为现场音频和视频广播创建rtsp客户端的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！