React Native：实时相机数据，无需保存和预览图像 [英] React native: Real time camera data without image save and preview

问题描述

我开始开发我的第一个非演示react-native应用程序。我希望它将是一个iOS / Android应用程序，但实际上我只专注于iOS。

I started working on my first non-demo react-native app. I hope it will be a iOS/Android app, but actually I'm focused on iOS only.

我实际上有一个问题。如何在不保存到相机胶卷的情况下从相机实时获取数据（base64，像素阵列等）。

I have a one problem actually. How can I get a data (base64, array of pixels, ...) in real-time from the camera without saving to the camera roll.

有此模块： https://github.com/lwansbrough/react-native-camera 但base64已弃用，对我无用，因为我希望向用户渲染处理过的图像（例如，更改图片颜色），而不是相机的真实图片，因为它会使用react-native-camera模块。

There is this module: https://github.com/lwansbrough/react-native-camera but base64 is deprecated and is useless for me, because I want a render processed image to user (change picture colors eg.), not the real picture from camera, as it does react-native-camera module.

（我知道如何与SWIFT代码进行通信，但是我不知道本机代码中的选项，我来自WebDev）

(I know how to communicate with SWIFT code, but I don't know what the options are in native code, I come here from WebDev)

非常感谢。

推荐答案

这可能不是最佳选择，但我一直在使用。如果有人可以提供更好的解决方案，我也将感谢您的帮助！

This may not be optimal but is what I have been using. If anyone can give a better solution, I would appreciate your help, too!

我的基本想法是简单地循环（但不简单，请参见下文）以yuv / rgb格式以最大分辨率拍摄静态图片，该分辨率相当快（正常曝光时间约为x0ms）并进行处理。基本上，您将设置链接到您的摄像机的AVCaptureStillImageOutput（遵循所有教程），然后将格式设置为kCVPixelFormatType_420YpCbCr8BiPlanarFullRange（如果您想使用YUV）或kCVPixelFormatType_32BGRA（如果您希望使用rgba），例如

My basic idea is simply to loop (but not simple for-loop, see below) taking still pictures in yuv/rgb format at max resolution, which is reasonably fast (~x0ms with normal exposure duration) and process them. Basically you will setup AVCaptureStillImageOutput that links to you camera (following tutorials everywhere) then set the format to kCVPixelFormatType_420YpCbCr8BiPlanarFullRange (if you want YUV) or kCVPixelFormatType_32BGRA(if you prefer rgba) like

bool usingYUVFormat = true;
NSDictionary *outputFormat = [NSDictionary dictionaryWithObject:
                [NSNumber numberWithInt:usingYUVFormat?kCVPixelFormatType_420YpCbCr8BiPlanarFullRange:kCVPixelFormatType_32BGRA]
                                           forKey:(id)kCVPixelBufferPixelFormatTypeKey];
[yourAVCaptureStillImageOutput setOutputSettings:outputFormat];

准备就绪后，您可以开始致电

When you are ready, you can start calling

AVCaptureConnection *captureConnection=[yourAVCaptureStillImageOutput connectionWithMediaType:AVMediaTypeVideo];
[yourAVCaptureStillImageOutput captureStillImageAsynchronouslyFromConnection:captureConnection  completionHandler:^(CMSampleBufferRef imageDataSampleBuffer, NSError *error) {
    if(imageDataSampleBuffer){
        CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(imageDataSampleBuffer);
        CVPixelBufferLockBaseAddress(imageBuffer, 0);
        // do your magic with the data buffer imageBuffer
        // use CVPixelBufferGetBaseAddressOfPlane(imageBuffer,0/1/2); to get each plane
        // use CVPixelBufferGetWidth/CVPixelBufferGetHeight to get dimensions
        // if you want more, please google
    }
}];

另外，使用NSNotificationCenter注册您的拍照操作并在处理完每帧后发布通知（可能会有所延迟，以限制吞吐量并降低功耗），因此循环将继续进行。

Additionally, use NSNotificationCenter to register your photo-taking action and post a notification after you have processed each frame (with some delay perhaps, to cap your through-put and reduce power consumption) so the loop will keep going.

一个快速的预防措施：Android同类产品差得多头痛。正如我在我的问题。我仍在寻找解决方案，但放弃了最大的希望。 JPEG图像太慢了。

A quick precaution: the Android counterpart is much worse a headache. Few hardware manufacturers implement api for max-resolution uncompressed photos but only 1080p for preview/video, as I have raised in my question. I am still looking for solutions but gave up most hope. JPEG images are just toooo slow.

这篇关于React Native：实时相机数据，无需保存和预览图像的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！