从Swift中的CVPixelBufferRef获取像素值 [英] Get pixel value from CVPixelBufferRef in Swift

问题描述

如何从CVPixelBufferRef中获取RGB（或任何其他格式）像素值？我已经尝试了很多方法，但还没有成功。

  func captureOutput（captureOutput：AVCaptureOutput！，didOutputSampleBuffer sampleBuffer：CMSampleBuffer！，fromConnection connection： AVCaptureConnection！）{
 
让pixelBuffer：CVPixelBufferRef = CMSampleBufferGetImageBuffer（sampleBuffer）！ 
 CVPixelBufferLockBaseAddress（pixelBuffer，0）
 let baseAddress = CVPixelBufferGetBaseAddress（pixelBuffer）
 
 
 //在此处获取单个像素值
 
 CVPixelBufferUnlockBaseAddress（ pixelBuffer，0）
} 
  

解决方案

baseAddress 是一个不安全的可变指针，或者更确切地说是 UnsafeMutablePointer< Void> 。将指针从 Void 转换为更具体的类型后，您可以轻松访问内存：

  //将基址转换为适当类型的安全指针
 let byteBuffer = UnsafeMutablePointer< UInt8>（baseAddress）
 
 //读取数据（返回类型为UInt8的值）
 let firstByte = byteBuffer [0] 
 
 //写入数据
 byteBuffer [3] = 90 
  

确保使用正确的类型（8,16或32位无符号整数）。这取决于视频格式。最有可能是8位。

缓冲格式更新：

你可以初始化 AVCaptureVideoDataOutput 实例时指定格式。您基本上可以选择：

• BGRA：蓝色，绿色，红色和alpha值存储在32位的单个平面每个整数


• 420YpCbCr8BiPlanarFullRange：两个平面，第一个包含具有Y（亮度）值的每个像素的字节，第二个包含像素组的Cb和Cr（色度）值


• 420YpCbCr8BiPlanarVideoRange：与420YpCbCr8BiPlanarFullRange相同，但Y值限制在16 - 235范围内（由于历史原因）

如果您对颜色值感兴趣并且速度（或者说最大帧速率）不是问题，那么请选择更简单的BGRA格式。否则采用一种更有效的原生视频格式。

如果你有两架飞机，你必须得到所需飞机的基地址（见视频格式示例）： / p>

视频格式示例

 让pixelBuffer ：CVPixelBufferRef = CMSampleBufferGetImageBuffer（sampleBuffer）！ 
 CVPixelBufferLockBaseAddress（pixelBuffer，0）
 let baseAddress = CVPixelBufferGetBaseAddressOfPlane（pixelBuffer，0）
 let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane（pixelBuffer，0）
 let byteBuffer = UnsafeMutablePointer< UInt8>（baseAddress） 
 
 //获取像素的亮度值（43,17）
 let luma = byteBuffer [17 * bytesPerRow + 43] 
 
 CVPixelBufferUnlockBaseAddress（pixelBuffer，0）
  

BGRA示例

 让pixelBuffer：CVPixelBufferRef = CMSampleBufferGetImageBuffer（sampleBuffer）！ 
 CVPixelBufferLockBaseAddress（pixelBuffer，0）
 let baseAddress = CVPixelBufferGetBaseAddress（pixelBuffer）
 let int32PerRow = CVPixelBufferGetBytesPerRow（pixelBuffer）
 let int32Buffer = UnsafeMutablePointer< UInt32>（baseAddress）
 
 //获取像素的BGRA值（43,41）
让luma = int32Buffer [17 * int32PerRow + 43] 
 
 CVPixelBufferUnlockBaseAddress（pixelBuffer，0）
  

How can I get the RGB (or any other format) pixel value from a CVPixelBufferRef? Ive tried many approaches but no success yet.

        func captureOutput(captureOutput: AVCaptureOutput!, didOutputSampleBuffer sampleBuffer: CMSampleBuffer!, fromConnection connection: AVCaptureConnection!) {

            let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
            CVPixelBufferLockBaseAddress(pixelBuffer, 0)
            let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)


    //Get individual pixel values here

    CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)
        }

解决方案

baseAddress is an unsafe mutable pointer or more precisely a UnsafeMutablePointer<Void>. You can easily access the memory once you have converted the pointer away from Void to a more specific type:

// Convert the base address to a safe pointer of the appropriate type
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// read the data (returns value of type UInt8)
let firstByte = byteBuffer[0]

// write data
byteBuffer[3] = 90

Make sure you use the correct type (8, 16 or 32 bit unsigned int). It depends on the video format. Most likely it's 8 bit.

Update on buffer formats:

You can specify the format when you initialize the AVCaptureVideoDataOutput instance. You basically have the choice of:

• BGRA: a single plane where the blue, green, red and alpha values are stored in a 32 bit integer each
• 420YpCbCr8BiPlanarFullRange: Two planes, the first containing a byte for each pixel with the Y (luma) value, the second containing the Cb and Cr (chroma) values for groups of pixels
• 420YpCbCr8BiPlanarVideoRange: The same as 420YpCbCr8BiPlanarFullRange but the Y values are restricted to the range 16 – 235 (for historical reasons)

If you're interested in the color values and speed (or rather maximum frame rate) is not an issue, then go for the simpler BGRA format. Otherwise take one of the more efficient native video formats.

If you have two planes, you must get the base address of the desired plane (see video format example):

Video format example

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0)
let bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(pixelBuffer, 0)
let byteBuffer = UnsafeMutablePointer<UInt8>(baseAddress)

// Get luma value for pixel (43, 17)
let luma = byteBuffer[17 * bytesPerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

BGRA example

let pixelBuffer: CVPixelBufferRef = CMSampleBufferGetImageBuffer(sampleBuffer)!
CVPixelBufferLockBaseAddress(pixelBuffer, 0)
let baseAddress = CVPixelBufferGetBaseAddress(pixelBuffer)
let int32PerRow = CVPixelBufferGetBytesPerRow(pixelBuffer)
let int32Buffer = UnsafeMutablePointer<UInt32>(baseAddress)

// Get BGRA value for pixel (43, 17)
let luma = int32Buffer[17 * int32PerRow + 43]

CVPixelBufferUnlockBaseAddress(pixelBuffer, 0)

这篇关于从Swift中的CVPixelBufferRef获取像素值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！