如何从AVCapturePhoto生成具有正确方向的UIImage? [英] How to generate an UIImage from AVCapturePhoto with correct orientation?

问题描述

我正在调用AVFoundation的委托方法来处理照片捕获，但是我很难将其生成的AVCapturePhoto转换为具有正确方向的UIImage.尽管下面的例程成功了，但我总是得到面向右的UIImage(UIImage.imageOrientation = 3).使用UIImage(data: image)时，我无法提供方向信息，尝试首先使用photo.cgImageRepresentation()?.takeRetainedValue()也是没有帮助的.请协助.

I am calling AVFoundation's delegate method to handle a photo capture, but I am having difficulty converting the AVCapturePhoto it generates into an UIImage with the correct orientation. Although the routine below is successful, I always get a right-oriented UIImage (UIImage.imageOrientation = 3). I have no way of providing an orientation when using the UIImage(data: image) and attempting to first use photo.cgImageRepresentation()?.takeRetainedValue() also doesn't help. Please assist.

在这里，图像定向至关重要，因为最终的图像将被馈送到Vision Framework工作流程中.

Image orientation is critical here as the resulting image is being fed to a Vision Framework workflow.

func photoOutput(_ output: AVCapturePhotoOutput, didFinishProcessingPhoto photo: AVCapturePhoto, error: Error?) {
    // capture image finished
    print("Image captured.")
    if let imageData = photo.fileDataRepresentation() {
        if let uiImage = UIImage(data: imageData){
            // do stuff to UIImage
        }
    }
}

更新1: 阅读Apple的照片捕捉编程指南(日期为iOS11)，我确实找到了我做错了的一件事:

UPDATE 1: Reading Apple's Photo Capture Programming Guide (out of date for iOS11), I did manage to find one thing I was doing wrong:

1. 在每个捕获调用(self.capturePhotoOutput.capturePhoto)上，必须与PhotoOutput对象建立连接，并更新其方向以匹配拍摄照片时设备的方向.为此，我创建了UIDeviceOrientation的扩展名并将其用于创建的snapPhoto()函数中，以调用捕获例程并等待didFinishProcessingPhoto委托方法被执行.我添加了代码快照，因为此处的代码示例定界符似乎无法正确显示它们.

1. On every capture call (self.capturePhotoOutput.capturePhoto) one must setup a connection with the PhotoOutput object and update its orientation to match the device's orientation at the moment the picture is taken. For doing that, I created an extension of UIDeviceOrientation and used it on the snapPhoto() function I created to call the capture routine and wait for the didFinishProcessingPhoto delegate method to be executed. I've added a snapshot of the code because the code sample delimiters here don't seem to be displaying them correctly.

更新2 链接到GitHub上的完整项目: https://github.com/agu3rra/Out-Loud

Update 2 Link to full project on GitHub: https://github.com/agu3rra/Out-Loud

推荐答案

最终更新: 我对该应用程序进行了一些实验，得出以下结论:

Final update: I ran some experiments with the app and came to the following conclusions:

1. kCGImagePropertyOrientation似乎并没有影响应用程序中捕获的图像的方向，并且仅当您每次要调用photoOutput连接时，它才随设备方向而变化. >方法.所以:

1. kCGImagePropertyOrientation doesn’t seem to influence the orientation of the captured image inside your application and it only varies with the device orientation if you update your photoOutput connection each time you are about to call the capturePhoto method. So:

func snapPhoto() {
    // prepare and initiate image capture routine

    // if I leave the next 4 lines commented, the intented orientation of the image on display will be 6 (right top) - kCGImagePropertyOrientation
    let deviceOrientation = UIDevice.current.orientation // retrieve current orientation from the device
    guard let photoOutputConnection = capturePhotoOutput.connection(with: AVMediaType.video) else {fatalError("Unable to establish input>output connection")}// setup a connection that manages input > output
    guard let videoOrientation = deviceOrientation.getAVCaptureVideoOrientationFromDevice() else {return}
    photoOutputConnection.videoOrientation = videoOrientation // update photo's output connection to match device's orientation

    let photoSettings = AVCapturePhotoSettings()
    photoSettings.isAutoStillImageStabilizationEnabled = true
    photoSettings.isHighResolutionPhotoEnabled = true
    photoSettings.flashMode = .auto
    self.capturePhotoOutput.capturePhoto(with: photoSettings, delegate: self) // trigger image capture. It appears to work only if the capture session is running.
}

在调试器上查看生成的图像，向我展示了如何生成它们，因此我可以推断出所需的旋转角度(UIImageOrientation)以使其竖直显示.换句话说:更新UIImageOrientation告诉您应该如何旋转图像，以便您以正确的方向观看它.所以我来到了下表:

Viewing the generated images on the debugger has shown me how they get generated, so I could infer the required rotation (UIImageOrientation) to get it displayed upright. In other words: updating UIImageOrientation tells how the image should be rotated in order for you to see it in the correct orientation. So I came to the following table:

我不得不将我的UIDeviceOrientation扩展名更新为一种不太直观的形式:

I had to update my UIDeviceOrientation extension to a rather unintuitive form:

extension UIDeviceOrientation {
    func getUIImageOrientationFromDevice() -> UIImageOrientation {
        // return CGImagePropertyOrientation based on Device Orientation
        // This extented function has been determined based on experimentation with how an UIImage gets displayed.
        switch self {
        case UIDeviceOrientation.portrait, .faceUp: return UIImageOrientation.right
        case UIDeviceOrientation.portraitUpsideDown, .faceDown: return UIImageOrientation.left
        case UIDeviceOrientation.landscapeLeft: return UIImageOrientation.up // this is the base orientation
        case UIDeviceOrientation.landscapeRight: return UIImageOrientation.down
        case UIDeviceOrientation.unknown: return UIImageOrientation.up
        }
    }
}

这是我现在的最终委托方法的外观.它将以预期的方向显示图像.

This is how my final delegate method looks now. It displays the image in the expected orientation.

func photoOutput(_ output: AVCapturePhotoOutput,
                                 didFinishProcessingPhoto photo: AVCapturePhoto,
                                 error: Error?)
{
    // capture image finished
    print("Image captured.")

    let photoMetadata = photo.metadata
    // Returns corresponting NSCFNumber. It seems to specify the origin of the image
    //                print("Metadata orientation: ",photoMetadata["Orientation"])

    // Returns corresponting NSCFNumber. It seems to specify the origin of the image
    print("Metadata orientation with key: ",photoMetadata[String(kCGImagePropertyOrientation)] as Any)

    guard let imageData = photo.fileDataRepresentation() else {
        print("Error while generating image from photo capture data.");
        self.lastPhoto = nil; self.controller.goToProcessing();
        return

    }

    guard let uiImage = UIImage(data: imageData) else {
        print("Unable to generate UIImage from image data.");
        self.lastPhoto = nil; self.controller.goToProcessing();
        return

    }

    // generate a corresponding CGImage
    guard let cgImage = uiImage.cgImage else {
        print("Error generating CGImage");self.lastPhoto=nil;return

    }

    guard let deviceOrientationOnCapture = self.deviceOrientationOnCapture else {
        print("Error retrieving orientation on capture");self.lastPhoto=nil;
        return

    }

    self.lastPhoto = UIImage(cgImage: cgImage, scale: 1.0, orientation: deviceOrientationOnCapture.getUIImageOrientationFromDevice())

    print(self.lastPhoto)
    print("UIImage generated. Orientation:(self.lastPhoto.imageOrientation.rawValue)")
    self.controller.goToProcessing()
}


func photoOutput(_ output: AVCapturePhotoOutput, 
                   willBeginCaptureFor resolvedSettings: AVCaptureResolvedPhotoSettings) 
                   {
    print("Just about to take a photo.")
    // get device orientation on capture
    self.deviceOrientationOnCapture = UIDevice.current.orientation
    print("Device orientation: \(self.deviceOrientationOnCapture.rawValue)")
}

这篇关于如何从AVCapturePhoto生成具有正确方向的UIImage?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！