如果已有新帧，如何跳过didReceiveFrame？ [英] How to skip didReceiveFrame when there is already a new frame?

问题描述

在webrtc应用程序中有一个名为didReceiveFrame的回调，当有一个要渲染的新帧时会调用该回调。它将新帧作为参数传递，并且可以从该帧中提取纹理。但是，如果由于某种原因主线程被延迟（想想断点，设备旋转，仍然忙着渲染等等），那么对于每个错过帧，将单独调用此回调。这有效地增加了相机捕获的内容和呈现的内容之间的延迟。

In a webrtc app there is a callback called didReceiveFrame, which is called when there is a new frame to be rendered. It passes the new frame as argument and one can extract a texture from this frame. However, if for some reason the main thread is delayed (think breakpoint, device rotation, still busy rendering, etc...) then this callback is called separately for every 'missed' frame. This effectively adds a delay between what the camera captures and what is rendered.

如何确保仅使用最新帧调用didReceiveFrame？或者我如何才能看到传递的帧是否是最新的（所以我可以跳过该函数）？

How can I make sure that didReceiveFrame is only called with the latest frame? Or how can I see if the passed frame is the latest (so I can skip the function)?

推荐答案

我从未找到过使用库执行此操作的方法，但我使用 NSDate 实现了我自己的简单检查。我在渲染器类中存储了一个属性，如下所示：

I never found a way to do this with the library, but I implemented my own simple check using an NSDate. I store a property in the renderer class like this:

@property (strong, nonatomic) NSDate* timeOfLastFrame;

我在我的 viewDidLoad 中初始化它：

self.timeOfLastFrame = [[NSDate alloc] initWithTimeIntervalSince1970:0.0];

我在我的渲染方法中使用它，如下所示：

and I use it in my render method like this:

- (void)renderFrame:(RTCI420Frame*)frame {
    if ([[NSDate date] timeIntervalSinceDate:self.timeOfLastFrame] > 1.0f/24.0f) { // or whatever fps you want
        self.timeOfLastFrame = [NSDate date];
        // do rendering
    }
}

1.0f / 24.0f part确定最大fps。在这种情况下，它不会每秒渲染超过24帧，这对于视频流来说相当不错，并且足够低以注意到性能提升。如果在1/24秒之前调用 renderFrame 方法，则它根本不执行任何操作。

The 1.0f/24.0f part determines the max fps. In this case, it won't render more than 24 frames per second, which is pretty decent for a video stream and low enough to notice a performance increase. If the renderFrame method is called before 1/24th of a second has passed it simply doesn't do anything.

注意：这样做意味着渲染不会是稳定的24fps，因为它不会跳过每个x帧，而是依赖于渲染时如果方法在前一个方法之后过早出现，则会调用该方法并简单地删除该方法。我在我的应用程序中没有注意到它，但是要记住这一点，特别是在使用低最大fps时。

Note: Doing it like this means the rendering won't be at a steady 24fps, because it doesn't skip every x frames, instead it relies on when the render method is called and simply drops a frame if it came too soon after the previous one. I don't notice it in my app, but it's something to keep in mind, especially when using a low max fps.

这篇关于如果已有新帧，如何跳过didReceiveFrame？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！