录制视频时将动态文本呈现到CVPixelBufferRef上 [英] Render dynamic text onto CVPixelBufferRef while recording video

问题描述

我正在使用AVCaptureVideoDataOutput和AVCaptureAudioDataOutput录制视频和音频，并且在captureOutput:didOutputSampleBuffer:fromConnection:委托方法中，我想在从视频连接接收的每个单独的样本缓冲区上绘制文本.文本每隔一帧都会变化(这是一个秒表标签)，我希望将其记录在所捕获的视频数据之上.

I'm recording video and audio using AVCaptureVideoDataOutput and AVCaptureAudioDataOutput and in the captureOutput:didOutputSampleBuffer:fromConnection: delegate method, I want to draw text onto each individual sample buffer I'm receiving from the video connection. The text changes with about every frame (it's a stopwatch label) and I want that to be recorded on top of the video data that's captured.

这是到目前为止我能想到的:

Here's what I've been able to come up with so far:

//1.
CVPixelBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);

//2.   
UIImage *textImage = [self createTextImage];
CIImage *maskImage = [CIImage imageWithCGImage:textImage.CGImage];

//3.
CVPixelBufferLockBaseAddress(pixelBuffer, 0); 
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
NSDictionary *options = [NSDictionary dictionaryWithObject:(__bridge id)colorSpace forKey:kCIImageColorSpace];
CIImage *inputImage = [CIImage imageWithCVPixelBuffer:pixelBuffer options:options];

//4.
CIFilter *filter = [CIFilter filterWithName:@"CIBlendWithMask"];
[filter setValue:inputImage forKey:@"inputImage"];
[filter setValue:maskImage forKey:@"inputMaskImage"];
CIImage *outputImage = [filter outputImage];
CVPixelBufferUnlockBaseAddress(pixelBuffer, 0);

//5.   
[self.renderContext render:outputImage toCVPixelBuffer:pixelBuffer bounds:[outputImage extent] colorSpace:CGColorSpaceCreateDeviceRGB()];

//6.
[self.pixelBufferAdaptor appendPixelBuffer:pixelBuffer withPresentationTime:timestamp];

1. 在这里，我抓住了像素缓冲区，就像馅饼一样.
2. 我使用核心图形将文本写到空白的UIImage中(createTextImage就是这样做.我能够验证此步骤是否有效；我将在其上绘制了文字的图像保存到了我的照片中.
3. 我从像素缓冲区创建CGImage.
4. 我为CIBlendWithMask制作了一个CIFilter，将输入图像设置为由原始像素缓冲区创建的图像，将输入蒙版设置为由图像上绘制了文字的图像制成的CIImage.
5. 最后，我将滤镜输出图像渲染到pixelBuffer上. CIContext是使用[CIContext contextWithOptions:nil];预先创建的.
6. 所有这些之后，我将像素缓冲区附加到具有适当时间戳记的pixelBufferAdaptor上.

1. Here I grab the pixel buffer, easy as pie.
2. I use core graphics to write text to a blank UIImage (that's what createTextImage does. I was able to verify that this step works; I saved an image with text drawn to it to my photos.
3. I create a CGImage from the pixel buffer.
4. I make a CIFilter for CIBlendWithMask, setting the input image as the one created from the original pixel buffer and the input mask as the CIImage made from the image with text drawn on it.
5. Finally, I render the filter output image to the pixelBuffer. The CIContext was created beforehand with [CIContext contextWithOptions:nil];.
6. After all that, I append the pixel buffer to my pixelBufferAdaptor with the appropriate timestamp.

在录制结束时保存的视频没有可见的变化，即没有在像素缓冲区上绘制遮罩图像.

The video that's saved at the end of recording has no visible changes to it i.e. no mask image has been drawn onto the pixel buffers.

有人知道我在哪里错吗?我已经坚持了好几天，任何帮助将不胜感激.

Anyone have any idea where I'm going wrong here? I've been stuck on this for days, any help would be so appreciated.

- (UIImage *)createTextImage {
    UIGraphicsBeginImageContextWithOptions(CGSizeMake(self.view.bounds.size.width, self.view.bounds.size.height), NO, 1.0);
    NSMutableAttributedString *timeStamp = [[NSMutableAttributedString alloc]initWithString:self.timeLabel.text attributes:@{NSForegroundColorAttributeName:self.timeLabel.textColor, NSFontAttributeName: self.timeLabel.font}];
    NSMutableAttributedString *countDownString = [[NSMutableAttributedString alloc]initWithString:self.cDownLabel.text attributes:@{NSForegroundColorAttributeName:self.cDownLabel.textColor, NSFontAttributeName:self.cDownLabel.font}];
    [timeStamp drawAtPoint:self.timeLabel.center];
    [countDownString drawAtPoint:self.view.center];
    UIImage *blank = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    return blank;
}

推荐答案

是否要执行以下操作?

Do you want to as below?

您应该使用CISourceOverCompositing，而不是使用CIBlendWithMask，请尝试以下操作:

Instead of using CIBlendWithMask, you should use CISourceOverCompositing, try this:

//4.
CIFilter *filter = [CIFilter filterWithName:@"CISourceOverCompositing"];
[filter setValue:maskImage forKey:kCIInputImageKey];
[filter setValue:inputImage forKey:kCIInputBackgroundImageKey];
CIImage *outputImage = [filter outputImage];

这篇关于录制视频时将动态文本呈现到CVPixelBufferRef上的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！