寻找速度最慢的视频编码器，将网络摄像头流式传输到ipad [英] Look for fastest video encoder with least lag to stream webcam streaming to ipad
问题描述
我正在寻找可以在html5视频标签中查看的网络摄像头流的最快速编码方式。我正在使用Pandaboard： http： //www.digikey.com/product-highlights/us/en/texas-instruments-pandaboard/686#tabs-2 的硬件。可以使用gstreamer，cvlc，ffmpeg。我会用它来驱动机器人，所以在视频流中需要最少的滞后。质量不一定很大，不需要音频。此外，这仅适用于一个客户端，因此带宽不是问题。迄今为止最好的解决方案是使用ffmpeg与mpjpeg给我大约1秒的延迟。任何更好的东西？
我已经被问过这么多次，所以我会尝试一下回答这一点，而不仅仅是为了mjpeg 。在系统中获得非常低的延迟需要一些系统工程的努力和对组件的理解。
我可以想到的一些简单的顶级调整是：
确保编解码器配置为最低延迟。编解码器将具有（特别是嵌入式系统编解码器）低延迟配置。启用它如果你使用H.264，这是最有用的。大多数人没有意识到，按照标准要求，H.264解码器在显示之前需要缓冲帧。 Qcif最多可达16个，720p可达5帧。这是第一帧的延迟很多。如果您不使用H.264仍然确保您没有启用B图片。这增加了获取第一张照片的延迟。
由于您使用mjpeg，我认为这不适用于您。
编码器还将具有速率控制延迟。 （调用init延迟或vbv buf大小）。将其设置为使您可以接受的质量的最小值。这也将减少延误。将其视为编码器和解码器之间的比特流缓冲区。如果使用的是x264，那将是vbv缓冲区大小。
一些简单的其他配置：尽可能少的使用图片（大内部周期）。
我的图片是巨大的，并增加了通过网络发送的延迟。这在端到端延迟在1秒以上的系统中可能不是很明显，但是当您设计需要100ms或更短的端到端延迟的系统时，这个和其他几个方面发挥作用。还要确保您使用低延迟音频编解码器aac-lc（而不是heaac）。
在你的情况下，为了降低延迟，我建议从mjpeg离开，至少使用mpeg4没有B图片（简单的配置文件）或最好的是H.264基线配置文件（x264给出了一个zerolatency选项）。您将获得较低延迟的简单原因是您将获得较低的比特率后缀编码以将数据发送出去，您可以进行完整的帧速率。如果您必须坚持使用mjpeg，那么您可以使用开放源代码组件，从编码解码器和系统中获得更多高级功能的支持。
另一方面是将内容传输到显示单元。如果可以使用udp，与tcp相比，它可以减少延迟时间，尽管它有时会因为网络条件而有损耗。你已经提到html5视频。我很好奇你是如何做现场直播html5视频标签。
还有其他方面也可以调整我将放在高级类别和要求系统工程师尝试各种各样的事情。
操作系统中的网络缓冲是什么？出于性能原因，操作系统还会在发送数据之前缓存数据。调整这一点，以获得性能和速度之间的良好平衡。您是否使用CR或VBR编码？虽然CBR对于低抖动来说非常棒，但如果编解码器提供了CBR，也可以使用封顶的vbr。
您的解码器可以开始解码部分帧吗？因此，在将数据提供给解码器之前，您不用担心成帧数据。只需要尽快将数据推送到解码器。
可以进行字段编码吗？
每当切片可以通过网络立即发送时，您是否可以使用回调进行切片编码？
>使用我在上述所有工作的100 ms延迟系统。某些功能可能在开源组件中不可用，但如果您真的需要它，并且很有热情，您可以继续执行它们。
编辑：
我意识到，对于ipad流媒体解决方案，您无法做很多上述工作，而且由于hls还可以实现延迟，因此存在限制。但是，我希望在其他情况下，当您需要任何低延迟系统时，这将证明是有用的。
I'm looking for the fastest way to encode a webcam stream that will be viewable in a html5 video tag. I'm using a Pandaboard: http://www.digikey.com/product-highlights/us/en/texas-instruments-pandaboard/686#tabs-2 for the hardware. Can use gstreamer, cvlc, ffmpeg. I'll be using it to drive a robot, so need the least amount of lag in the video stream. Quality doesn't have to be great and it doesn't need audio. Also, this is only for one client so bandwidth isn't an issue. The best solution so far is using ffmpeg with a mpjpeg gives me around 1 sec delay. Anything better?
I have been asked this many times so I will try and answer this a bit generically and not just for mjpeg. Getting very low delays in a system requires a bit of system engineering effort and also understanding of the components.
Some simple top level tweaks I can think of are:
Ensure the codec is configured for the lowest delay. Codecs will have (especially embedded system codecs) a low delay configuration. Enable it. If you are using H.264 it's most useful. Most people don't realize that by standard requirements H.264 decoders need to buffer frames before displaying it. This can be upto 16 for Qcif and upto 5 frames for 720p. That is a lot of delay in getting the first frame out. If you do not use H.264 still ensure you do not have B pictures enabled. This adds delay to getting the first picture out.
Since you are using mjpeg, I don't think this is applicable to you much.
Encoders will also have a rate control delay. (Called init delay or vbv buf size). Set it to the smallest value that gives you acceptable quality. That will also reduce the delay. Think of this as the bitstream buffer between encoder and decoder. If you are using x264 that would be the vbv buffer size.
Some simple other configurations: Use as few I pictures as possible (large intra period). I pictures are huge and add to the delay to send over the network. This may not be very visible in systems where end to end delay is in the range of 1 second or more but when you are designing systems that need end to end delay of 100ms or less, this and several other aspects come into play. Also ensure you are using a low latency audio codec aac-lc (and not heaac).
In your case to get to lower latencies I would suggest moving away from mjpeg and use at least mpeg4 without B pictures (Simple profile) or best is H.264 baseline profile (x264 gives a zerolatency option). The simple reason you will get lower latency is that you will get lower bitrate post encoding to send the data out and you can go to full framerate. If you must stick to mjpeg you have close to what you can get without more advanced features support from the codec and system using the open source components as is.
Another aspect is the transmission of the content to the display unit. If you can use udp it will reduce latency quite a lot compared to tcp, though it can be lossy at times depending on network conditions. You have mentioned html5 video. I am curious as to how you are doing live streaming to a html5 video tag.
There are other aspects that can also be tweaked which I would put in the advanced category and requires the system engineer to try various things out
What is the network buffering in the OS? The OS also buffers data before sending it out for performance reasons. Tweak this to get a good balance between performance and speed.
Are you using CR or VBR encoding? While CBR is great for low jitter you can also use capped vbr if the codec provides it.
Can your decoder start decoding partial frames? So you don't have to worry about framing the data before providing it to the decoder. Just keep pushing the data to the decoder as soon as possible.
Can you do field encoding? Halves the time from frame encoding before getting the first picture out.
Can you do sliced encoding with callbacks whenever a slice is available to send over the network immediately?
In sub 100 ms latency systems that I have worked in all of the above are used. Some of the features may not be available in open source components but if you really need it and are enthusiastic you could go ahead and implement them.
EDIT: I realize you cannot do a lot of the above for a ipad streaming solution and there are limitations because of hls also to the latency you can achieve. But I hope it will prove useful in other cases when you need any low latency system.
这篇关于寻找速度最慢的视频编码器，将网络摄像头流式传输到ipad的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
