在 Swift 中从 CVPixelBufferRef 获取像素值 [英] Get pixel value from CVPixelBufferRef in Swift

问题描述

如何从 CVPixelBufferRef 获取 RGB(或任何其他格式)像素值?我尝试了很多方法，但都没有成功.

How can I get the RGB (or any other format) pixel value from a CVPixelBufferRef? Ive tried many approaches but no success yet.

func captureOutput(captureOutput: AVCaptureOutput!,
                   didOutputSampleBuffer sampleBuffer: CMSampleBuffer!,
                   fromConnection connection: AVCaptureConnection!) {
  let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
                CVPixelBufferLockBaseAddress(pixelBuffer, 0)
  let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)

  //Get individual pixel values here

  CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
}

推荐答案

baseAddress 是一个不安全的可变指针，或者更准确地说是一个 UnsafeMutablePointer.一旦将指针从 Void 转换为更具体的类型，您就可以轻松访问内存:

baseAddress is an unsafe mutable pointer or more precisely a UnsafeMutablePointer<Void>. You can easily access the memory once you have converted the pointer away from Void to a more specific type:

// Convert the base address to a safe pointer of the appropriate type
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// read the data (returns value of type UInt8)
let firstByte = byteBuffer[0]

// write data
byteBuffer[3] = 90

确保您使用正确的类型(8、16 或 32 位无符号整数).这取决于视频格式.很可能是 8 位.

Make sure you use the correct type (8, 16 or 32 bit unsigned int). It depends on the video format. Most likely it's 8 bit.

缓冲区格式更新:

您可以在初始化 AVCaptureVideoDataOutput 实例时指定格式.您基本上可以选择:

You can specify the format when you initialize the AVCaptureVideoDataOutput instance. You basically have the choice of:

• BGRA:单个平面，其中蓝色、绿色、红色和 alpha 值分别存储在 32 位整数中
• 420YpCbCr8BiPlanarFullRange:两个平面，第一个平面包含每个像素的字节和 Y(亮度)值，第二个包含像素组的 Cb 和 Cr(色度)值
• 420YpCbCr8BiPlanarVideoRange:与 420YpCbCr8BiPlanarFullRange 相同，但 Y 值限制在 16 – 235 范围内(由于历史原因)

如果您对颜色值和速度(或者更确切地说是最大帧速率)感兴趣，那么请选择更简单的 BGRA 格式.否则采用一种更高效的原生视频格式.

If you're interested in the color values and speed (or rather maximum frame rate) is not an issue, then go for the simpler BGRA format. Otherwise take one of the more efficient native video formats.

如果你有两个平面，你必须得到所需平面的基地址(见视频格式示例):

If you have two planes, you must get the base address of the desired plane (see video format example):

视频格式示例

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// Get luma value for pixel (43, 17)
let luma = byteBuffer[17 * bytesPerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

BGRA 示例

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)
let int32PerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
let int32Buffer = UnsafeMutablePointer<UInt32>(baseAddress)

// Get BGRA value for pixel (43, 17)
let luma = int32Buffer[17 * int32PerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

这篇关于在 Swift 中从 CVPixelBufferRef 获取像素值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！