Android 10 (api 29) camera2 api 回归与广角相机 [英] Android 10 (api 29) camera2 api regression with wide-angle camera

查看:112
本文介绍了Android 10 (api 29) camera2 api 回归与广角相机的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我在我的相机应用程序中使用了camera2 api,该应用程序专为Google Pixel 3 XL 设计.该设备有两个前置摄像头(广角和普通).多亏了多摄像头功能,我可以同时访问两个物理摄像头设备,我的应用程序具有在这两个摄像头之间切换的功能.在我最近升级到 Android 10 之前,我可以准确地看到两种不同的结果,但现在我的广角捕捉帧与普通相机具有几乎相同的 FOV(视野).所以,同样的代码,同样的 apk 在 Android 9 广角捕获结果是宽的,正如预期的那样,而且在 Andoird 10 升级之后 - 广角和普通相机显示几乎相同的 FOV.

这是一个代码片段,用于演示我如何初始化两个摄像头和捕获预览:

MainActivity.kt

 private val surfaceReadyCallback = object: SurfaceHolder.Callback {覆盖 fun surfaceChanged(p0: SurfaceHolder?, p1: Int, p2: Int, p3: Int) { }覆盖乐趣surfaceDestroyed(p0:SurfaceHolder?){}覆盖乐趣surfaceCreated(p0:SurfaceHolder?){//从活动/片段中获取两个输出目标val surface1 = surfaceView1.holder.surfaceval surface2 = surfaceView2.holder.surfaceval dualCamera = findShortLongCameraPair(cameraManager)!!val outputTargets = DualCameraOutputs(null, mutableListOf(surface1), mutableListOf(surface2))//打开逻辑摄像头,配置输出并创建会话createDualCameraSession(cameraManager, dualCamera, targets = outputTargets) { session ->val requestTemplate = CameraDevice.TEMPLATE_PREVIEWval captureRequest = session.device.createCaptureRequest(requestTemplate).apply {arrayOf(surface1,surface2).forEach { addTarget(it) }}.建造()session.setRepeatingRequest(captureRequest, null, null)}}}有趣的 openDualCamera(cameraManager: CameraManager,双摄像头:双摄像头,执行器:执行器 = SERIAL_EXECUTOR,回调:(CameraDevice)->单元) {cameraManager.openCamera(dualCamera.logicalId,执行者,对象:CameraDevice.StateCallback() {覆盖乐趣 onOpened(设备:CameraDevice){回调(设备)}覆盖乐趣 onError(device: CameraDevice, error: Int) = onDisconnected(device)覆盖乐趣 onDisconnected(device: CameraDevice) = device.close()})}有趣的 createDualCameraSession(cameraManager: CameraManager,双摄像头:双摄像头,目标:DualCameraOutputs,执行器:执行器 = SERIAL_EXECUTOR,回调:(CameraCaptureSession)->单元) {//创建 3 组输出配置:一组用于逻辑相机,以及//每个物理相机一个.val outputConfigsLogical = targets.first?.map { OutputConfiguration(it) }val outputConfigsPhysical1 = targets.second?.map {OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId1) } }val outputConfigsPhysical2 = targets.third?.map {OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId2) } }val outputConfigsAll = arrayOf(outputConfigsLogical、outputConfigsPhysical1、outputConfigsPhysical2).filterNotNull().flatten()val sessionConfiguration = SessionConfiguration(SessionConfiguration.SESSION_REGULAR,outputConfigsAll,执行器,对象:CameraCaptureSession.StateCallback() {覆盖乐趣 onConfigured(session: CameraCaptureSession) = callback(session)覆盖乐趣 onConfigureFailed(session: CameraCaptureSession) = session.device.close()})openDualCamera(cameraManager, dualCamera, executor = executor) {it.createCaptureSession(sessionConfiguration)}}

DualCamera.kt 助手类

data class DualCamera(val logicalId: String, val physicalId1: String, val physicalId2: String)fun findDualCameras(manager: CameraManager, faces: Int? = null): Array{val dualCameras = ArrayList()manager.cameraIdList.map {对(manager.getCameraCharacteristics(它),它)}.筛选 {面对 == 空 ||it.first.get(CameraCharacteristics.LENS_FACING) == 面对}.筛选 {it.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)!!.contains(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA)}.forEach {val physicalCameras = it.first.physicalCameraIds.toTypedArray()for (idx1 in 0 until physicalCameras.size) {for (idx2 in (idx1 + 1) until physicalCameras.size) {双摄像头.add(双摄像头(it.second, physicalCameras[idx1], physicalCameras[idx2]))}}}返回 dualCameras.toTypedArray()}有趣的 findShortLongCameraPair(manager: CameraManager, faces: Int? = null): DualCamera?{返回 findDualCameras(经理,面对).地图 {val features1 = manager.getCameraCharacteristics(it.physicalId1)val features2 = manager.getCameraCharacteristics(it.physicalId2)val focusLengths1 = features1.get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F)val focusLengths2 = features2.get(CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F)val focusLengthsDiff1 = focusLengths2.max()!!- focusLengths1.min()!!val focusLengthsDiff2 = focusLengths1.max()!!- focusLengths2.min()!!如果(focalLengthsDiff1 

您可以在附加的屏幕截图中看到结果,左上角的 FOV 比运行在 Android 10 上的同一相机宽得多

这是 Android 10 的已知回归吗?有没有人注意到类似的行为?

解决方案

我的理解:我在我的 Pixel 3 上遇到了同样的问题.看起来广角相机的框架在组合之前已经在 HAL 层中被裁剪了.实际上FOV并不完全相同,因为左右相机之间存在一些差异.但是,广角相机的默认缩放级别似乎根据焦距而变化.

但是我找不到任何关于它的官方文档.在 Android 10 中,它声称改进了物理相机的融合:

I'm using camera2 api in my camera app designed specifically for Google Pixel 3 XL. This device has two front facing cameras (wide-angle and normal). Thanks to multi-camera feature, I can access both physical camera devices simultaneously, and my app has a feature to toggle between those two cameras. Up until my recent upgrade to Android 10, I could accurately see two distinct results, but now my wide-angle capture frame has pretty much the same FOV (Field of View) as the normal camera one. So, the same code, same apk on Android 9 wide-angle capture result is wide, as expected, and after Andoird 10 upgrade - wide and normal cameras show practically identical FOV.

Here is a code snippet to demonstrate how I initialize both cameras and capture preview:

MainActivity.kt

 private val surfaceReadyCallback = object: SurfaceHolder.Callback {
        override fun surfaceChanged(p0: SurfaceHolder?, p1: Int, p2: Int, p3: Int) { }
        override fun surfaceDestroyed(p0: SurfaceHolder?) { }

        override fun surfaceCreated(p0: SurfaceHolder?) {

            // Get the two output targets from the activity / fragment
            val surface1 = surfaceView1.holder.surface  
            val surface2 = surfaceView2.holder.surface 

            val dualCamera = findShortLongCameraPair(cameraManager)!!
            val outputTargets = DualCameraOutputs(
                null, mutableListOf(surface1), mutableListOf(surface2))

            //Open the logical camera, configure the outputs and create a session
            createDualCameraSession(cameraManager, dualCamera, targets = outputTargets) { session ->

                val requestTemplate = CameraDevice.TEMPLATE_PREVIEW
                val captureRequest = session.device.createCaptureRequest(requestTemplate).apply {
                    arrayOf(surface1, surface2).forEach { addTarget(it) }
                }.build()

                session.setRepeatingRequest(captureRequest, null, null)
            }
        }
    }


    fun openDualCamera(cameraManager: CameraManager,
                       dualCamera: DualCamera,
                       executor: Executor = SERIAL_EXECUTOR,
                       callback: (CameraDevice) -> Unit) {

        cameraManager.openCamera(
            dualCamera.logicalId, executor, object : CameraDevice.StateCallback() {
                override fun onOpened(device: CameraDevice) { callback(device) }

                override fun onError(device: CameraDevice, error: Int) = onDisconnected(device)
                override fun onDisconnected(device: CameraDevice) = device.close()
            })
    }

    fun createDualCameraSession(cameraManager: CameraManager,
                                dualCamera: DualCamera,
                                targets: DualCameraOutputs,
                                executor: Executor = SERIAL_EXECUTOR,
                                callback: (CameraCaptureSession) -> Unit) {

        // Create 3 sets of output configurations: one for the logical camera, and
        // one for each of the physical cameras.
        val outputConfigsLogical = targets.first?.map { OutputConfiguration(it) }
        val outputConfigsPhysical1 = targets.second?.map {
            OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId1) } }
        val outputConfigsPhysical2 = targets.third?.map {
            OutputConfiguration(it).apply { setPhysicalCameraId(dualCamera.physicalId2) } }

        val outputConfigsAll = arrayOf(
            outputConfigsLogical, outputConfigsPhysical1, outputConfigsPhysical2)
            .filterNotNull().flatten()

        val sessionConfiguration = SessionConfiguration(SessionConfiguration.SESSION_REGULAR,
            outputConfigsAll, executor, object : CameraCaptureSession.StateCallback() {
                override fun onConfigured(session: CameraCaptureSession) = callback(session)
                override fun onConfigureFailed(session: CameraCaptureSession) = session.device.close()
            })


        openDualCamera(cameraManager, dualCamera, executor = executor) {
           it.createCaptureSession(sessionConfiguration)
        }
    }

DualCamera.kt Helper Class

data class DualCamera(val logicalId: String, val physicalId1: String, val physicalId2: String)

fun findDualCameras(manager: CameraManager, facing: Int? = null): Array<DualCamera> {
    val dualCameras = ArrayList<DualCamera>()

    manager.cameraIdList.map {
        Pair(manager.getCameraCharacteristics(it), it)
    }.filter {
        facing == null || it.first.get(CameraCharacteristics.LENS_FACING) == facing
    }.filter {
        it.first.get(CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES)!!.contains(
            CameraCharacteristics.REQUEST_AVAILABLE_CAPABILITIES_LOGICAL_MULTI_CAMERA)
    }.forEach {
        val physicalCameras = it.first.physicalCameraIds.toTypedArray()
        for (idx1 in 0 until physicalCameras.size) {
            for (idx2 in (idx1 + 1) until physicalCameras.size) {
                dualCameras.add(DualCamera(
                    it.second, physicalCameras[idx1], physicalCameras[idx2]))
            }
        }
    }

    return dualCameras.toTypedArray()
}

fun findShortLongCameraPair(manager: CameraManager, facing: Int? = null): DualCamera? {

    return findDualCameras(manager, facing).map {
        val characteristics1 = manager.getCameraCharacteristics(it.physicalId1)
        val characteristics2 = manager.getCameraCharacteristics(it.physicalId2)

        val focalLengths1 = characteristics1.get(
            CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F)
        val focalLengths2 = characteristics2.get(
            CameraCharacteristics.LENS_INFO_AVAILABLE_FOCAL_LENGTHS) ?: floatArrayOf(0F)

        val focalLengthsDiff1 = focalLengths2.max()!! - focalLengths1.min()!!
        val focalLengthsDiff2 = focalLengths1.max()!! - focalLengths2.min()!!

        if (focalLengthsDiff1 < focalLengthsDiff2) {
            Pair(DualCamera(it.logicalId, it.physicalId1, it.physicalId2), focalLengthsDiff1)
        } else {
            Pair(DualCamera(it.logicalId, it.physicalId2, it.physicalId1), focalLengthsDiff2)
        }

        // Return only the pair with the largest difference, or null if no pairs are found
    }.sortedBy { it.second }.reversed().lastOrNull()?.first
}

And you can see the result on the attached screenshot, the top left corner one has much wider FOV than the same camera but running on Android 10

Is this a known regression with Android 10? Has anyone noticed similar behavior?

解决方案

My understanding: I came across the same problem on my Pixel 3. It seems that the wide angle camera's frame has been cropped in the HAL layer before combination. Actually the FOV is not totally the same, as there is a little disparity between left and right camera. However, the default zoom level of wide camera seems to change according to the focal length.

But I could not find any official documentation about it. In Android 10, it claims improved the fusing of physical cameras: https://developer.android.com/about/versions/10/features#multi-camera

Solution:

If you wish to access the raw data from the wide angle front camera, you can create 2 camera sessions for both physical cameras instead of a single session for the logical camera.

Updated:

You can use the setPhysicalCameraKey to reset the zoom level https://developer.android.com/reference/android/hardware/camera2/CaptureRequest.Builder#setPhysicalCameraKey(android.hardware.camera2.CaptureRequest.Key%3CT%3E,%20T,%20java.lang.String)

这篇关于Android 10 (api 29) camera2 api 回归与广角相机的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆