从相机以OpenCV Mat格式捕获静止图像 [英] Capturing a still image from camera in OpenCV Mat format

问题描述

我正在开发iOS应用程序，并尝试使用捕获会话从相机获取静止图像快照，但是无法将其成功转换为OpenCV Mat.

I am developing an iOS application and trying to get a still image snapshot from the camera using capture session but I'm unable to convert it successfully to an OpenCV Mat.

使用以下代码创建静止图像输出:

The still image output is created using this code:

- (void)createStillImageOutput;
{
    // setup still image output with jpeg codec
    self.stillImageOutput = [[AVCaptureStillImageOutput alloc] init];
    NSDictionary *outputSettings = [NSDictionary dictionaryWithObjectsAndKeys:AVVideoCodecJPEG, AVVideoCodecKey, nil];
    [self.stillImageOutput setOutputSettings:outputSettings];
    [self.captureSession addOutput:self.stillImageOutput];

    for (AVCaptureConnection *connection in self.stillImageOutput.connections) {
        for (AVCaptureInputPort *port in [connection inputPorts]) {
            if ([port.mediaType isEqual:AVMediaTypeVideo]) {
                self.videoCaptureConnection = connection;
                break;
            }
        }
        if (self.videoCaptureConnection) {
            break;
        }
    }
    NSLog(@"[Camera] still image output created");
}

然后尝试使用以下代码捕获静止图像:

And then attempting to capture a still image using this code:

[self.stillImageOutput captureStillImageAsynchronouslyFromConnection:self.videoCaptureConnection
                                                   completionHandler:
 ^(CMSampleBufferRef imageSampleBuffer, NSError *error)
 {
     if (error == nil && imageSampleBuffer != NULL)
     {
         NSData *jpegData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageSampleBuffer];
     }

我需要一种基于缓冲区中的像素数据创建OpenCV Mat的方法. 我尝试使用从OpenCV摄像机类相机此处:

I need a way to create an OpenCV Mat based on the pixel data in the buffer. I have tried creating a Mat using this code which I have taken from OpenCV video camera class camera here:

    CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(imageBuffer, 0);

    void* bufferAddress;
    size_t width;
    size_t height;
    size_t bytesPerRow;

    CGColorSpaceRef colorSpace;
    CGContextRef context;

    int format_opencv;

    OSType format = CVPixelBufferGetPixelFormatType(imageBuffer);
    if (format == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {

        format_opencv = CV_8UC1;

        bufferAddress = CVPixelBufferGetBaseAddressOfPlane(imageBuffer, 0);
        width = CVPixelBufferGetWidthOfPlane(imageBuffer, 0);
        height = CVPixelBufferGetHeightOfPlane(imageBuffer, 0);
        bytesPerRow = CVPixelBufferGetBytesPerRowOfPlane(imageBuffer, 0);

    } else { // expect kCVPixelFormatType_32BGRA

        format_opencv = CV_8UC4;

        bufferAddress = CVPixelBufferGetBaseAddress(imageBuffer);
        width = CVPixelBufferGetWidth(imageBuffer);
        height = CVPixelBufferGetHeight(imageBuffer);
        bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer);

    }

    cv::Mat image(height, width, format_opencv, bufferAddress, bytesPerRow);

但是它无法在CVPixelBufferGetWidth或CVPixelBufferGetHeight调用上获取图像的实际高度和宽度，因此创建Mat失败.

but it fails to get the actual height and width of the image on the CVPixelBufferGetWidth or CVPixelBufferGetHeight call and so the creation of Mat fails.

我知道我可以使用以下代码基于像素数据创建UIImage:

I'm aware that I can create a UIImage based on the pixel data using this code:

             UIImage* newImage = [UIImage imageWithData:jpegData];

但是我更喜欢直接构造一个CvMat，就像在OpenCV CvVideoCamera类中那样，因为我只对在OpenCV中处理图像感兴趣，并且我不想浪费时间再次转换并且不会冒失去质量或方向出现问题(无论如何，OpenCV提供的UIImagetoCV转换功能会导致我的内存泄漏而不释放内存).

But I prefer to construct a CvMat directly as it is the case in the OpenCV CvVideoCamera class because I'm interested in handling the image in OpenCV only and I don't want to consume time converting again and not risk losing quality or having issues with orientation (anyway the UIImagetoCV conversion function provided by OpenCV is causing me memory leaks and not freeing the memory).

请告知我如何将图像作为OpenCV Mat获取. 预先感谢.

Please advise how can I get the image as an OpenCV Mat. Thanks in advance.

推荐答案

我已经找到解决问题的方法.

I have found the solution for my problem.

解决方案是:

通过在OpenCv Camera类中重写此方法:"createVideoPreviewLayer"

by overriding this method in the OpenCv Camera Class : "createVideoPreviewLayer"

它应该看起来像这样:

- (void)createVideoPreviewLayer;
{

    self.parentView.layer.sublayers = nil;
    if (captureVideoPreviewLayer == nil) {
        captureVideoPreviewLayer = [[AVCaptureVideoPreviewLayer alloc]
        initWithSession:self.captureSession];
    }

    if (self.parentView != nil) {
        captureVideoPreviewLayer.frame = self.parentView.bounds;
        captureVideoPreviewLayer.videoGravity =
        AVLayerVideoGravityResizeAspectFill;
        [self.parentView.layer addSublayer:captureVideoPreviewLayer];
    }
    NSLog(@"[Camera] created AVCaptureVideoPreviewLayer");
}

您应将此行添加到"createVideoPreviewLayer"方法中，以解决 问题:

you should add this line to the "createVideoPreviewLayer" method will solve the problem :

self.parentView.layer.sublayers = nil;

，您需要使用暂停方法而不是停止方法

这篇关于从相机以OpenCV Mat格式捕获静止图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！