为什么AVSampleBufferDisplayLayer停止显示取自AVCaptureVideoDataOutput的委托的CMSampleBuffers？ [英] Why AVSampleBufferDisplayLayer stops showing CMSampleBuffers taken from AVCaptureVideoDataOutput's delegate?
问题描述
我想用AVSampleBufferDisplayLayer显示一些CMSampleBuffer，但是在显示第一个样本后它会冻结。
I want to display some CMSampleBuffer's with the AVSampleBufferDisplayLayer, but it freezes after showing the first sample.
我从AVCaptureVideoDataOutputSampleBuffer委托中获取样本缓冲区：
I get the samplebuffers from the AVCaptureVideoDataOutputSampleBuffer delegate:
-(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection
{
CFRetain(sampleBuffer);
[self imageToBuffer:sampleBuffer];
CFRelease(sampleBuffer);
}
将它们放入向量
-(void) imageToBuffer: (CMSampleBufferRef )source{
//buffers is defined as: std::vector<CMSampleBufferRef> buffers;
CMSampleBufferRef newRef;
CMSampleBufferCreateCopy(kCFAllocatorDefault, source, &newRef);
buffers.push_back(newRef);
}
然后尝试通过AVSampleBufferDisplayLayer（在另一个ViewController中）显示它们
Then try to show them via AVSampleBufferDisplayLayer (in another ViewController)
AVSampleBufferDisplayLayer * displayLayer = [[AVSampleBufferDisplayLayer alloc] init];
displayLayer.bounds = self.view.bounds;
displayLayer.position = CGPointMake(CGRectGetMidX(self.displayOnMe.bounds), CGRectGetMidY(self.displayOnMe.bounds));
displayLayer.videoGravity = AVLayerVideoGravityResizeAspectFill;
displayLayer.backgroundColor = [[UIColor greenColor] CGColor];
[self.view.layer addSublayer:displayLayer];
self.view.autoresizingMask = UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
dispatch_queue_t queue = dispatch_queue_create("My queue", DISPATCH_QUEUE_SERIAL);
[displayLayer setNeedsDisplay];
[displayLayer requestMediaDataWhenReadyOnQueue:queue
usingBlock:^{
while ([displayLayer isReadyForMoreMediaData]) {
if (samplesKey < buffers.size()) {
CMSampleBufferRef buf = buffers[samplesKey];
[displayLayer enqueueSampleBuffer:buffers[samplesKey]];
samplesKey++;
}else
{
[displayLayer stopRequestingMediaData];
break;
}
}
}];
但它会显示第一个样本然后冻结，并且什么都不做。
but it shows the first sample then freezes, and does nothing.
我的视频数据输出设置如下：
And my video data output settings are as follows:
//set up our output
self.videoDataOutput = [[AVCaptureVideoDataOutput alloc] init];
dispatch_queue_t queue = dispatch_queue_create("VideoQueue", DISPATCH_QUEUE_SERIAL);
[_videoDataOutput setSampleBufferDelegate:self queue:queue];
[_videoDataOutput setVideoSettings:[NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:kCVPixelFormatType_32BGRA],(id)kCVPixelBufferPixelFormatTypeKey,
nil]];
推荐答案
我在同一个环境中遇到了这个问题，尝试过从AVCaptureVideoDataOutput获取输出并将其显示在AVSampleDisplay图层中。
I came across this problem in the same context, trying to take the output from AVCaptureVideoDataOutput and display it in a AVSampleDisplay layer.
如果您的帧按显示顺序出现，那么修复非常简单，只需在CMSampleBufferRef上设置显示器立即标记。
If your frames come out in display order, then the fix is very easy, just set the display immediately flag on the CMSampleBufferRef.
获取委托返回的样本缓冲区然后...
Get the sample buffer returned by the delegate and then...
CFArrayRef attachments = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer, YES);
CFMutableDictionaryRef dict = (CFMutableDictionaryRef)CFArrayGetValueAtIndex(attachments, 0);
CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanTrue);
如果您的帧以编码器顺序（不是显示顺序）出现，那么CMSampleBuffer上的时间戳需要进行零偏置和重新缓冲，使第一帧时间戳等于时间0.
If your frames come out in encoder order (not display order), then the time stamps on the CMSampleBuffer need to be zero biased and restamped such that the first frames timestamp is equal to time 0.
double pts = CMTimeGetSeconds(CMSampleBufferGetPresentationTimeStamp(sampleBuffer));
// ptsStart is equal to the first frames presentationTimeStamp so playback starts from time 0.
CMTime presentationTimeStamp = CMTimeMake((pts-ptsStart)*1000000,1000000);
CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer, presentationTimeStamp);
更新：
我遇到了一种情况，当我使用零偏差方法时，某些视频仍然无法播放，我进一步调查了。正确的答案似乎是使用你打算玩的第一个框架中的PTS。
I ran into a situation where some video still wasn't playing smoothly when I used the zero bias method and I investigated further. The correct answer seems to be using the PTS from the first frame you intend to play.
我的答案就在这里，但我会发布它这里也是。
My answer is here, but I will post it here, too.
设置AVSampleBufferDisplayLayer呈现样本缓冲区的速率
时基需要设置为时间戳的显示时间戳（pts）你打算解码的第一帧。我通过从所有后续pts中减去初始pts并将Timebase设置为0来将第一帧的pts索引为0.无论出于何种原因，这对某些视频都不起作用。
The Timebase needs to be set to the presentation time stamp (pts) of the first frame you intend to decode. I was indexing the pts of the first frame to 0 by subtracting the initial pts from all subsequent pts and setting the Timebase to 0. For whatever reason, that didn't work with certain video.
你想要这样的东西（在调用解码前称为）：
You want something like this (called before a call to decode):
CMTimebaseRef controlTimebase;
CMTimebaseCreateWithMasterClock( CFAllocatorGetDefault(), CMClockGetHostTimeClock(), &controlTimebase );
displayLayer.controlTimebase = controlTimebase;
// Set the timebase to the initial pts here
CMTimebaseSetTime(displayLayer.controlTimebase, CMTimeMake(ptsInitial, 1));
CMTimebaseSetRate(displayLayer.controlTimebase, 1.0);
设置CMSampleBuffer的PTS ......
Set the PTS for the CMSampleBuffer...
CMSampleBufferSetOutputPresentationTimeStamp(sampleBuffer, presentationTimeStamp);
并且可能确保没有立即显示....
And maybe make sure display immediately isn't set....
CFDictionarySetValue(dict, kCMSampleAttachmentKey_DisplayImmediately, kCFBooleanFalse);
WWDC 2014 Session 513会对此进行了简要介绍。
This is covered very briefly in WWDC 2014 Session 513.
这篇关于为什么AVSampleBufferDisplayLayer停止显示取自AVCaptureVideoDataOutput的委托的CMSampleBuffers？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
