音视频同步，TS MPEG2；H264/AVC，了解Handbrake中的PTS [英] Audio/video synchronization, TS MPEG2;H264/AVC, understanding PTS in Handbrake
问题描述
同步一直让我着迷，或者准确地说:为什么媒体播放器可以同步查看 .ts，而重新组合的解复用音频 + 视频不同步.
Synchronization has always fascinated me, or to be precise: why a .ts can be viewed in sync by media players, while the demuxed audio+video reassembled is out of sync.
所以我试图理解这一点，以及可以做些什么来防止它.
So I'm trying to understand this, and what can be done to prevent it.
我已阅读以下内容:https://trac.handbrake.fr/wiki/LibHandBrakeSync和 sync.c 的来源(也可在 wiki 上找到)
I've read the following: https://trac.handbrake.fr/wiki/LibHandBrakeSync and the source of sync.c (also available on the wiki)
BitStreamTools 也写了一篇关于这个主题的 Theory 101(但我无法链接，因为我是新用户，抱歉)
BitStreamTools have written a Theory 101 on the subject also (but I can't link as I'm a new user, sorry)
虽然我认为我对 PCR/PTS 的理解(在概念上)是正确的，但我很难遵循 Handbrake 出色的 A/V 同步论文.
While I thought my understanding of PCR/PTS was (conceptually) right, I'm having a hard time following handbrake's excellent A/V sync paper.
我的问题是:对 a/v 同步是否有某种直观的解释(可以是简短的、短的或长的、长的)?虽然我知道如果音频或视频 pts 损坏(不连续?)，可以从 PCR 重新计算 PTS，但手刹似乎不依赖于此，而是依赖于它的内部 PTS.0, += 1/fps (~=5), 10, 15, ....
My question is this: is there a somewhat intuitive (it can be brief, short or longer, as long) explanation of a/v synchronization? While I know that one can recalculate PTS from PCR if audio or video pts is corrupted (discontinuity?), handbrake does not seem to rely on this, but on it's internal PTS. 0, += 1/fps (~=5), 10, 15, ....
是否可以通过修复所有音频和视频 PTS 值来重新计算 pts 偏移量并更正 .ts(二进制)(并使用相同的偏移量倾斜所有 DTS，因此播放器不会用完帧")，可以这么说)，因此有一个可以解复用的 .ts，然后孤立的轨道是同步的(如果重新组合在一起)?
Would it be possible to recalculate the pts offsets and correct the .ts (binary) by fixing all audio and video PTS values (and skewing all DTS with the same offset, so the player doesn't "run out of frames", so to speak), and thus have a .ts which can be demuxed, and the isolated tracks then be in sync (if put back together)?
还是无法通过使用 PCR 重新计算给定 .ts 中的所有 PTS 值来修复?虽然我知道某些帧/音频可能会在广播中损坏，因此无法正确呈现，但我会留下处理此问题(例如，如果视频已损坏并具有相应的音频部分，则将其删除，如果音频包损坏等)到以后，为了讨论起见，我假设所有帧都完好无损.(但是 PTS 值总是正确的，还是什么?)
Or would it not be possible to fix by using PCR to recalculate all PTS values in a given .ts? While I understand that some frames/audio might be damaged in broadcast so it can not be presented correctly, I'll leave the handling of this (such as removing the video if it's damaged and has corresponding audio part, inserting x ms silence if the audio package is damaged etc.) to later, and for the sake of discussion I'll presume all frames are intact. (But then the PTS values would always be correct though, or what?)
附录:我对手刹 A/V 纸的看法是这样的:在预期"为 100 时，偏移量计算为视频 pts (100) - 音频 pts (0) - 内部 PTS，以使音频达到相同的呈现时间，从而使 pts 偏移量为 99.在 105 时，偏移量将是 105-5 = 100，而不是 99，但我们继续使用 99 作为偏移量，因为不需要重新计算(100-99 = 1. 1/fps <100ms).在 150 处，随着视频 pts 的减少，而不是增加，将再次计算 pts 偏移量...
Appendix: My take on the handbrake A/V paper is this: At "expected" 100, the offset is calculated as video pts (100) - audio pts (0) - the internal PTS, to bring the audio up to the same presentation time, thus giving a pts offset of 99. at 105 the offset would be 105-5 = 100, not 99, but we proceed to use 99 as offset since there's no need to recalculate (100-99 = 1. 1/fps < 100ms). At 150, the pts offset is calculated again as the video pts is decreasing, as opposed to increasing...
我几乎可以肯定我完全错了，但有人可以指出我正确的方向吗?
I'm almost positive I'm complete wrong about this, but can someone point me in the right direction, please?
- 乔希
推荐答案
音视频同步的概念要深得多.我会推荐的第一个阅读是以下论文.
The concept of Audio Video synchronization is much deeper. The first reading i would recommed is the following paper.
http://downloads.bbc.co.uk/rd/pubs/reports/1996-02.pdf
我不会在这里重复所有内容 - 但基本上，每个编码器都会记录时间戳并将其标记在相应的音频和视频上.稍后，当解码器播放它时，它会做两件事 - 一是确保解码器自己的时钟被编码器的时钟奴役"，二是确保每张图片都显示在屏幕上，并且音频帧在相应的时间准确地呈现给扬声器时间发生.这是音频与视频保持同步的唯一和最佳方式.这些时间戳称为 PTS/DTS 值，其分辨率为 90 kHz 时钟.
I won't repeat everything here - but essentially, every encoder records timestamps and stamps it on the respective Audio and Video. Later on, when decoder plays it, it does two things - one, ensures that decoder's own clock is "enslaved" with encoder's clock, and two it ensures that every picture is presented on the screen and audio frame presented to speaker exactly when that respective time occurs. This is only and best way that audio remains in synchronization with video. These timestamps are called PTS/DTS values which are of resolution of 90 kHz clock.
了解随着时间的推移时钟会发生偏差，但由于只引用了确切的时间，因此解码器以完全相同的时间顺序播放.
Understand that over time clocks skew but since only the exact time is referenced, decoder playout exactly in same time order.
现在主要的问题仍然是解码器的时钟需要保持在编码器时钟的控制/同步中.在 MPEG 中做的第一件事是在 27 MHz 下使用更高的精度(高 300 倍).此外，这需要在中间的任何传输路径期间保持一致.(这称为时钟恢复过程).
Now the major concern remains is that decoder's clock needs to remain in control/synchronization of encoder's clock. The first thing done in MPEG is using a higher precision at 27 MHz, (300 times higher). Further, this needs to remain consistent during any transmission path in the middle. (this is called clock recovery process).
下面是另外几篇很好的论文，解释了时钟恢复/同步过程的工作原理.
Below are another couple of good paper that explains how clock recovery/synchronization process works.
https://www.soe.ucsc.edu/sites/default/files/technical-reports/UCSC-CRL-98-04.pdf
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.86.1016&rep=rep1&type=pdf
这篇最终论文将所有内容都很好地结合在一起.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.50.975&rep=rep1&type=pdf
This final paper puts every thing together much nicely.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.50.975&rep=rep1&type=pdf
请记住 - 基于 PCR 和 PTS/DTS 的音频视频同步使数字电视广播非常严格，并且与 Internet 流媒体中使用的任何其他流媒体方法大不相同.这对于使其 24x7 流式传输功能至关重要.
Remember - the PCR and PTS/DTS based audio video synchronization is what make Digital TV broadcast is very stringent and is far different from any other streaming methods used in Internet streaming. This is crucial to make it 24x7 streaming to function.
这篇关于音视频同步，TS MPEG2；H264/AVC，了解Handbrake中的PTS的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
