Gstreamer:在RTP流中暂停/恢复视频 [英] Gstreamer: Pausing/resuming video in RTP streams

问题描述

我正在构建一个从网络源接收两个RTP流的gstreamer管道:

I'm constructing a gstreamer pipeline that receives two RTP streams from an networked source:

1. ILBC音频流+相应的RTCP流
2. H263视频流+相应的RTCP流

所有内容都放入一个gstreamer管道中，因此它将使用来自两个流的RTCP来同步音频/视频.到目前为止，我已经提出了这一点(使用gst-launch进行原型制作):

Everything is put into one gstreamer pipeline so it will use the RTCP from both streams to synchronize audio/video. So far I've come up with this (using gst-launch for prototyping):

gst-launch -vvv  gstrtpbin name=rtpbin
  udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-2000" port=40000 ! rtpbin.recv_rtp_sink_0
  rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink
  udpsrc port=40001 ! rtpbin.recv_rtcp_sink_0
  rtpbin.send_rtcp_src_0 ! udpsink port=40002 sync=false async=false

  udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMU,encoding-params=(string)1,octet-align=(string)1" port=60000 rtpbin.recv_rtp_sink_1
  rtpbin. ! rtppcmudepay ! autoaudiosink
  udpsrc port=60001 ! rtpbin.recv_rtcp_sink_1 
  rtpbin.send_rtcp_src_1 ! udpsink port=60002 sync=false async=false

如果联网源同时发送视频和音频，则此管道可以很好地工作.如果稍后暂停视频流，则当联网源恢复视频流时，gstreamer仍将播放音频，甚至开始播放视频.

This pipeline works well if the networked source starts out with sending both video and audio. If the videostream is paused later on, gstreamer will still playback audio and even will start playing back the video when the networked source resumes the video stream.

但是，我的问题是，如果网络源仅以音频流开始(稍后可能会添加视频)，则管道似乎会暂停/冻结，直到视频流也开始为止.

My problem is however that if the networked source starts out with only an audio stream (video might be added later on), the pipeline seems to pause/freeze until the video stream starts as well.

由于视频在我的应用程序中是可选的(用户可以随意添加/删除)，因此我可以通过任何方式挂接"videotestsrc"，该视频将提供某种后备视频数据以保持没有网络视频数据时管道正在运行?

Since video is optional (and can be added/removed at will by the user) in my application, is there any way I can hook up for instance an 'videotestsrc' that will provide some kind of fallback video data to keep the pipeline running when there is no networked video data?

我已经尝试过使用"videotestsrc"和一个名为"videomixer"的东西进行尝试，但是我认为混音器仍然需要两个流都处于活动状态.任何反馈都将不胜感激！

I've tried experimenting with 'videotestsrc' and a thing called 'videomixer' but I think that mixer still requires both streams to be alive. Any feedback is greatly appreciated!

推荐答案

我展示了一个简单的函数，可通过更改垃圾箱来恢复暂停.在下面的示例中，我提供了动态地动态更改目标bin的逻辑.我相信，这不会完全阻止正在寻求的管道.类似的逻辑可以用于src bin.在这里，您可以删除网络源容器和相关的解码器/多路分配器容器，并添加videotestsrc容器.

I present a simple function for pause resume by changing bins. In the following example I provide the logic to change destination bin on the fly dynamically. This shall not completely stop the pipeline which is what you seek I believe. A similar logic could be used for src bins. Here you may remove your network source bin and related decoder/demux bins and add videotestsrc bins.

private static void dynamic_bin_replacement(Pipeline pipe, Element src_bin, Element dst_bin_new, Element dst_bin_old) {
pipe.pause();
src_bin.unlink(dst_bin_old);                     
pipe.remove(dst_bin_old);
pipe.add(dst_bin_new);
dst_bin_new.syncStateWithParent();
src_bin.link(dst_bin_new);
pipe.ready();                    
pipe.play();
}

您可能要尝试的其他逻辑是"PADLOCKING".请看看以下帖子

The other logic you may want to try is "PADLOCKING". Please take a look at the following posts

http://cgit.freedesktop.org /gstreamer/gstreamer/tree/docs/design/part-block.txt

和

http://web.archiveorange.com/archive/v/8yxpz7FmOlGqxVYtkPb4

和

添加和删除音频源往返GStreamer管道

更新

1. 尝试使用输出选择器和输入选择器箱，因为它们似乎是更好的选择.我发现它们是最可靠的，并给他们带来了极大的好运.我分别使用 fakesink 或 fakesrc 作为选择器的另一端.

1. Try output-selector and input-selector bins as they seem to be better alternative. I found them most reliable and have had immense luck with them. I use fakesink or fakesrc respectively as the other end of the selector.

阀门箱是另一个选择，甚至不需要 fakesink 或 fakesrc 箱.它也非常可靠.

valve bin is another alternative that I found doesn't even need fakesink or fakesrc bins. It is also extremely reliable.

还有媒体文件源的正确状态转换顺序

Also the correct state transition order for media file source

NULL->就绪->暂停->播放(向上)

播放->暂停->就绪->空(向下)

在上面的示例中，我的订单应该更正，其中ready()应该在pause()之前.我也倾向于认为应该在null()状态之后而不是在pause()之后执行取消链接.我没有尝试过这些更改，但从理论上讲，它们应该可以工作.

My order in the above example should be corrected where ready() should come before pause(). Also I would tend to think un-linking should be performed after null() state and not after pause(). I haven't tried these changes but theoretically they should work.

有关详细信息，请参见以下链接

See the following link for detailed info

http ://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-states.txt?h = BRANCH-RELEASE-0_10_19

这篇关于Gstreamer:在RTP流中暂停/恢复视频的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！