CIImage回到CMSampleBuffer [英] CIImage back to CMSampleBuffer

问题描述

我使用 AVAssetWriter 和 CMSampleBuffer 数据（来自视频，音频输入）记录视频（.mp4文件）。

在记录我要处理帧的同时，我正在将 CMSampleBuffer 转换为 CIImage 并对其进行处理。

但是如何用我新处理的图像缓冲区更新 CMSampleBuffer CIImage ？

  func captureOutput（_输出：AVCaptureOutput，didOutput sampleBuffer： CMSampleBuffer，来自连接：AVCaptureConnection）{
如果输出== videoOutput {
让imageBuffer：CVPixelBuffer = CMSampleBufferGetImageBuffer（sampleBuffer）！ 
 let ciimage：CIImage = CIImage（cvPixelBuffer：imageBuffer）
 ... //我的代码来处理CIImage（例如添加增强现实）
 //但是如何将其转换回CMSampleBuffer ？ 
 //因为AVAssetWriterInput可以在文件中编码视频/音频，所以需要CMSampleBuffer 
 ... 
} 
 ... 
} 
  

解决方案

您需要使用CIContext的 render（_：to：bounds：colorSpace：） 方法。

然后，您可以使用以下命令从CVPixelBuffer创建CMSampleBuffer： CMSampleBufferCreateReadyWithImageBuffer（_：_：_：_：_：_：

出于效率原因，您可能需要使用CVPixelBuffer池，Apple AVCamPhotoFilter示例代码中显示了一个示例。特别是，请参见 RosyCIRenderer类。。 p>

也请参阅此答案，可能会对您有所帮助将CIFilter应用于视频文件并将其保存

I record video (.mp4 file) using AVAssetWriter with CMSampleBuffer data (from video, audio inputs).

While recording I want to process frames, I'm converting CMSampleBuffer to CIImage and processing it.

but how to update CMSampleBuffer with my new processed image buffer from CIImage?

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
    if output == videoOutput {
       let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)!
       let ciimage: CIImage = CIImage(cvPixelBuffer: imageBuffer)
       ... // my code to process CIImage (for example add augmented reality)
       // but how to convert it back to CMSampleBuffer?
       // because AVAssetWriterInput to encode video/audio in file needs CMSampleBuffer
       ...
    } 
    ...
}

解决方案

You need to render your CIImage into a CVPixelBuffer, using CIContext's render(_:to:bounds:colorSpace:) method.

Then you can create a CMSampleBuffer from the CVPixelBuffer using e.g. CMSampleBufferCreateReadyWithImageBuffer(_:_:_:_:_:)

You may need to use a pool of CVPixelBuffer for efficiency reasons, an example of this is shown in Apple's AVCamPhotoFilter sample code. In particular, see the RosyCIRenderer class.

Also see this answer which may help you Applying a CIFilter to a Video File and Saving it

这篇关于CIImage回到CMSampleBuffer的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！