编写自定义DirectShow RTSP / RTP源推送过滤器 - 来自实时来源的时间戳数据 [英] Writing custom DirectShow RTSP/RTP Source push filter - timestamping data coming from live sources

问题描述

我正在编写自定义的DirectShow源推送过滤器，它应该从视频服务器接收RTP数据并将它们推送到呈现器。我写了一个CVideoPushPin类，它继承自CSourceStream和CVideoReceiverThread类，它是从视频服务器接收RTP包的线程的包装。接收者线程本质上有三件事：

$ ul

接收原始的RTP包，并收集接收者报告所需的一些数据。

$ b

组合框架，将它们复制到缓冲区，并将有关它们的信息存储到256
元素队列中，定义如下：

  struct queue_elem {
 char * start; //指向缓冲区中的一个帧
 int length; //数据的长度
 REFERENCE_TIME recvTime; //接收帧的时间戳（流时间）
}; 
 
 struct data {
 struct queue_elem queue [QUEUE_LENGTH]; 
 int qWrIdx; 
 int qRdIdx; 
 HANDLE mutex; 
}; 
  

每个接收到的帧都有当前流的时间戳

  p-> StreamTime（refTime）; 
 REFERENCE_TIME rt = refTime.GetUnits（）; 
  

问题是我不确定我该如何为FillBuffer方法中的每个MediaSample设置时间戳。我尝试了几种方法，但回放要么停止，要么太慢。
目前FillBuffer方法如下所示：

  REFERENCE_TIME thisFrameStartTime，thisFrameEndTime; 
 //确定缓冲区中是否有至少4帧
 if（noOfFrames> = 4）
 {
 currentQe = m_myData.queue [m_myData.qRdIdx ++]; //取当前帧描述
 if（m_myData.qRdIdx> = QUEUE_LENGTH）
 {
 m_myData.qRdIdx = 0; 
} 
 nextQe = m_myData.queue [m_myData.qRdIdx];如果（currentQe.length> 0）
 {
 memcpy（pData，currentQe.start，currentQe.length）; 
 
 pSample-> SetActualDataLength（currentQe.length）; 
 CRefTime refTime; 
 m_pFilter-> StreamTime（refTime）; 
 REFERENCE_TIME rt; 
 rt = refTime.GetUnits（）; 
 pSample-> GetTime（& thisFrameStartTime，& thisFrameEndTime）; 
 thisFrameEndTime = thisFrameStartTime +（nextQe.recvTime  -  currentQe.recvTime）; 
 pSample-> SetTime（& thisFrameStartTime，& thisFrameEndTime）; 
} 
} 
 else 
 {
 pSample-> SetActualDataLength（0）; 
 
 
 
 $ b $ p 
 $ b 
在这种情况下，我注意到队列中的项目数量非常快（由于某些原因，FillBuffer方法不能快速地提取数据），结果是播放视频时延迟。有没有人有一个想法，当从实时来源接收数据时，我应该如何做时间戳？
解决方案
渲染器将绘制帧时图形的流时间到达示例对象上的时间戳。如果我正确地读取了你的代码，你在到达时用流时间来标记它们，所以它们总是迟到渲染。音频渲染器对此有些困惑：如果音频渲染器正在提供图形时钟，则会将当前的流时间报告为当前正在播放的任何样本，并且这将导致一些不期望的时间行为。 > 
 
 
 
 
 
您希望在将来设置一个时间，以允许通过图形的延迟和过滤器中的任何缓冲。尝试设置一个时间可能300毫秒到未来（流时间现在+ 300毫秒）。
 
 
你想在帧之间保持一致，所以不要根据时间戳在每一帧的到达时间。使用每帧的RTP时间戳，并将第一个的基线设置为将来300ms; （rtp  -  rtp_at_baseline）+ dshow baseline（以适当的单位转换）。
 
 
您需要以相同的方式为音频和视频流添加时间戳，但是，如果我记得，RTP时间戳在每个流中都有不同的基线，所以您需要使用RTCP数据包将RTP时间戳转换为（绝对）NTP时间，然后使用您的初始值将NTP转换为directshow基线（基线NTP =现在的dshow流时间+ 300ms）。
 
 
 $ G 
I'm writing custom DirectShow source push filter which is supposed to receive RTP data from video server and push them to the renderer. I wrote a CVideoPushPin class which inherits from CSourceStream and CVideoReceiverThread class which is a wrapper for a thread that receive RTP packets from video server. The receiver thread essentially does three things:


receives raw RTP packets and collects some data that is needed for Receiver Reports
assembles frames, copies them to the buffer and stores information about them into 256 
element queue, which is defined as follows:
struct queue_elem {
   char *start; // Pointer to a frame in a buffer
   int length; // Lenght of data
   REFERENCE_TIME recvTime; // Timestamp when the frame was received (stream time)
};

struct data {
   struct queue_elem queue[QUEUE_LENGTH];
   int qWrIdx;
   int qRdIdx;
HANDLE mutex;
};

every received frame is timestamped with current stream time  
p->StreamTime(refTime);
REFERENCE_TIME rt = refTime.GetUnits();



The problems is that I'm not sure how do I have to set timestamps for every MediaSample in FillBuffer method. I tried several ways, but the playback either stops or it is too slow.
Currently the FillBuffer method looks like this:
   REFERENCE_TIME thisFrameStartTime, thisFrameEndTime;
// Make sure if there are at least 4 frames in the buffer
    if(noOfFrames >= 4)
    {   
        currentQe = m_myData.queue[m_myData.qRdIdx++]; //Take current frame description     
        if(m_myData.qRdIdx >= QUEUE_LENGTH)
        {
            m_myData.qRdIdx = 0;
        }           
        nextQe = m_myData.queue[m_myData.qRdIdx]; //Take next frame description
        if(currentQe.length > 0)
        {
            memcpy(pData, currentQe.start, currentQe.length);               

             pSample->SetActualDataLength(currentQe.length);                
            CRefTime refTime;
            m_pFilter->StreamTime(refTime);
            REFERENCE_TIME rt;
            rt = refTime.GetUnits();
            pSample->GetTime(&thisFrameStartTime, &thisFrameEndTime);
            thisFrameEndTime = thisFrameStartTime + (nextQe.recvTime - currentQe.recvTime);
            pSample->SetTime(&thisFrameStartTime, &thisFrameEndTime);   
        }
    }
    else 
    {
        pSample->SetActualDataLength(0);
    }
In this case I noticed that the number of items in the queue increases very quickly (for some reason FillBuffer method cannot pull out data fast enough), and the result is increasing delay when playing video. Does anybody have a idea how should I do the timestamping when receiving data from live sources?
解决方案
The renderer will draw the frames when the graph's stream time reaches the timestamp on the sample object. If I read your code correctly, you are timestamping them with the stream time at arrival, so they will always be late at rendering. This is confused somewhat by the audio renderer: if the audio renderer is providing the graph's clock, then it will report the current stream time to be whatever sample it is currently playing, and that is going to cause some undesirable time behaviour.

You want to set a time in the future, to allow for the latency through the graph and any buffering in your filter. Try setting a time perhaps 300ms into the future (stream time now + 300ms).
You want to be consistent between frames, so don't timestamp them based on the arrival time of each frame. Use the RTP timestamp for each frame, and set the baseline for the first one to be 300ms into the future; subsequent frames are then (rtp - rtp_at_baseline) + dshow baseline (with appropriate unit conversions.
You need to timestamp the audio and the video streams in the same way, using the same baseline. However, if I remember, RTP timestamps have a different baseline in each stream, so you need to use the RTCP packets to convert RTP timestamps to (absolute) NTP time, and then convert NTP to directshow using your initial baseline (baseline NTP = dshow stream time now + 300ms).
G

                        
这篇关于编写自定义DirectShow RTSP / RTP源推送过滤器 - 来自实时来源的时间戳数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！