在目标 C 中创建用于实时音频和视频广播的 rtsp 客户端 [英] Creating a rtsp client for live audio and video broadcasting in objective C

问题描述

我正在尝试创建一个实时广播音频和视频的 RTSP 客户端.我修改了链接 http://www.gdcl.co.uk/downloads.htm<上的 iOS 代码/a> 并能够正确地将视频广播到服务器.但是现在我在广播音频部分时遇到了问题.在链接示例中，代码的编写方式是将视频数据写入文件，然后从文件中读取数据并将 NALU 的视频数据包上传到 RTSP 服务器.

I am trying to create a RTSP client which live broadcast Audio and Video. I modified the iOS code at link http://www.gdcl.co.uk/downloads.htm and able to broadcast the Video to server properly. But now i am facing issues in broadcasting the audio part. In the link example the code is written in such a way that it writes the Video data to file and than reads the data from the file and upload the NALU's video packets to RTSP server.

对于音频部分，我不知道如何继续.现在我尝试的是从麦克风获取音频缓冲区，然后通过添加 RTP 标头和 ALU 将其直接广播到服务器.有人可以让我知道是否有更好的方法来实现这一点并使用 lip sycn 音频/视频.

For Audio part i am not sure how to proceed on it. Right now what i have tried is that get the audio buffer from mic and than broadcast it to the server directly by adding RTP headers and ALU.. but This approach is not properly working as Audio starts lagging behind and lag increases with time. Can someone let me know if there is some better approach to achieve this and with lip sycn audio/video.

推荐答案

您是否在客户端丢失任何数据包?如果是这样，您需要留下空间".如果您收到数据包 1,2,3,4,6,7，您需要为丢失的数据包(5)留出空间.

Are you losing any packets on the client? If so, you need to leave "space." If you receive packet 1,2,3,4,6,7, You need to leave space for the missing packet (5).

另一种可能性是所谓的时钟漂移问题.您的客户端和服务器上的时钟(水晶)彼此不完全同步.
这可能是由环境、温度变化等引起的.

The other possibility is a what is known as a clock drift problem. The clock (crystal) on your client and server are not perfectly in sync with each other.
This can be caused by environment, temperature changes, etc.

假设在一个完美的世界中，您的服务器正在以 48000 Hz 的频率生成 20 毫秒的音频样本.您的客户端正在使用 48000 hz 的采样率播放它们.实际上，您的客户端和服务器并不完全是 48000hz.您的服务器可能是 48000.001，而您的客户端可能是 47999.9998.因此，您的服务器可能比您的客户端交付得更快，反之亦然.您可能会过快地消耗数据包而导致缓冲区运行不足，或者滞后太远并溢出客户端缓冲区.在您的情况下，听起来客户端播放速度太慢并且慢慢落后于服务器.你可能每分钟只延迟几毫秒，但问题会继续存在，它看起来像 1970 年代对口型功夫电影.

Let's say in a perfect world your server is producing audio samples 20ms audio samples at 48000 hz. Your client is playing them back using a sample rate of 48000 hz. Realistically your client and server are not exactly 48000hz. Your server might be 48000.001 and your client might be 47999.9998. So your server might be delivering faster than your client or vise versa. You would either consume packets too fast and under run the buffer or lag too far behind and overflow the client buffer. In your case, it sounds like the client is playing back too slow and slowly lagging behind the server. You might only lag a couple milliseconds per minute but the issue will keep continuing and it will look like a 1970s lip synced Kung Fu movie.

在其他设备中，通常有一个公共时钟线来保持同步.例如，摄像机时钟、MIDI 时钟.多轨录音机时钟.

In other devices, there is often a common clock line to keep things in sync. For example, Video camera clocks, midi clocks. multitrack recorder clocks.

当您通过 IP 传送数据时，客户端和服务器之间没有共享的公共时钟.因此，您的问题涉及在不同设备之间同步时钟.我已经使用这种通用方法成功解决了这个问题:

When you deliver data over IP, there is no common clock shared between a client and server. So your issue concerns syncing clocks between disparate devices with no. I have successfully solved this problem using this general approach:

• A) 让客户端计算一段时间内传入的数据包的速率.
• B) 让客户端计算数据包被消耗(回放)的速率.
• C) 根据 A 和 B 调整客户端的采样率.

因此您的客户要求您调整播放的采样率.所以是的，你玩得更快或更慢.请注意，播放速率的变化将非常微妙.您可以将采样率设置为 48000.0001 hz 而不是 48000 hz.人类无法检测到音高的差异，因为它只会导致音高的一小部分差异.我给出了一个非常简化的方法的解释.在开发此类控制系统时，还必须考虑许多其他细微差别和边缘情况.你不只是设置它然后忘记它.您需要一个控制系统来管理播放.

So your client requires that you adjust the sample rate of the playback. So yes you play it faster or slower. Note that the playback rate change will be very very subtle. You might set the sample rate to be 48000.0001 hz instead of 48000 hz. The difference in pitch would be undetectable by humans as it would only cause a fraction a cent difference in pitch. I gave an explanation of a very simplified approach. There many other nuances and edge cases that must be considered when developing such a control system. You don't just set it and forget it. You need a control system to manage the playback.

一个有趣的测试来证明这一点是使用两个具有完全相同文件的设备.长录音(比如 3 小时)是最好的.同时启动它们.播放 3 小时后，您会发现一个领先另一个.

An interesting test to demonstrate this is to take two devices with the exact same file. A long recording (say 3 hours) is best. Start them at the same time. After 3 hours of playback, you will notice that one is ahead of the other.

这篇文章解释了 流式传输音频和视频并非易事.

这篇关于在目标 C 中创建用于实时音频和视频广播的 rtsp 客户端的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！