使用AV Player访问单个帧 [英] Accesing Individual Frames using AV Player

问题描述

在最近的项目中，我必须使用AV Foundation单独访问视频的所有帧.另外，如果可能的话，可以随机访问它们(如数组)

In a recent project, I have to access all frames of my video individually using AV Foundation. Also, if possible to acess them randomly (like an array)

我试图研究这个问题，但没有得到任何有用的信息.

I tried to research the question but I didn't get anything useful.

注意:是否有任何有用的文档来熟悉AV Foundation?

Note: Is there any useful documentation to get familiar with the AV Foundation ?

推荐答案

您可以使用AVAssetReader依次枚举视频帧，如下所示:

You can enumerate the frames of your video serially using AVAssetReader, like this:

let asset = AVAsset(URL: inputUrl)
let reader = try! AVAssetReader(asset: asset)

let videoTrack = asset.tracksWithMediaType(AVMediaTypeVideo)[0]

// read video frames as BGRA
let trackReaderOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings:[String(kCVPixelBufferPixelFormatTypeKey): NSNumber(unsignedInt: kCVPixelFormatType_32BGRA)])

reader.addOutput(trackReaderOutput)
reader.startReading()

while let sampleBuffer = trackReaderOutput.copyNextSampleBuffer() {
    print("sample at time \(CMSampleBufferGetPresentationTimeStamp(sampleBuffer))")
    if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
        // process each CVPixelBufferRef here
        // see CVPixelBufferGetWidth, CVPixelBufferLockBaseAddress, CVPixelBufferGetBaseAddress, etc
    }
}

随机访问更为复杂.您可以使用AVPlayer + AVPlayerItemVideoOutput随时使用copyPixelBufferForItemTime，从t随时获取帧.答案，但其精妙之处在于您如何选择t.

Random access is more complicated. You could use an AVPlayer + AVPlayerItemVideoOutput to get frames from any time t, using copyPixelBufferForItemTime, as described in this answer, but the subtlety lies in how you choose that t.

如果要以均匀的间隔对视频进行采样，那很容易，但是如果要降落在串行AVAssetReader代码所见的相同帧/演示时间戳记上，则可能必须预处理文件 with AVAssetReader，以构建帧号->演示时间戳地图.如果您通过使用AVAssetReaderTrackOutput中的nil输出设置来跳过解码，则可以很快.

If you want to sample the video at uniform intervals, then that's easy, but if you want to land on the same frames/presentation time stamps that the serial AVAssetReader code sees, then you will probably have to preprocess the file with AVAssetReader, to build a frame number -> presentation timestamp map. This can be fast if you skip decoding by using nil output settings in AVAssetReaderTrackOutput.

这篇关于使用AV Player访问单个帧的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！