如何使用Android CameraX ImageAnalysis提高帧频？ [英] How to increase frame rate with Android CameraX ImageAnalysis?

问题描述

我正在研究新的CameraX API，以了解从当前的Camera2系统进行切换的可行性。

I'm investigating the new CameraX API, in relation to how viable it might be to switch over from our current Camera2 system.

在我们的Camera2系统中，我们使用OpenGL表面从PreviewCaptureSession捕获帧，并且在大多数设备上我们都达到了稳定的30fps图像处理速度，其中有些能够

In our Camera2 system, we use an OpenGL surface to capture the frames from a PreviewCaptureSession, and we are hitting consistent 30fps image processing speeds across most devices, with some able to hit 60fps with AutoExposure settings enabled.

CameraX没有提供接近该速度的任何东西，我不确定设置中是否缺少它。

CameraX isn't giving anything near that speed and I'm not sure if its something I've missed in setup.

我已经设置了CameraX和ImageAnalysis的测试示例，但是对于通过的图像数量，我得到了锁定的帧速率。

I've setup the test examples for CameraX and ImageAnalysis, but I'm getting locked frame rates for the number of images that comes through.

例如，我可以将分辨率设置为低至320x240，最高可以达到1920x960，两者都将以（似乎有上限）16fps出现。

For example, I could set the resolution as low as 320x240 up to 1920x960 and both will come out at a (seemingly capped) 16fps.

何时我添加了一个预览用例以使其旁边运行，并设置enableTorch（true），ImageAnalysis用例将突然开始变得越来越像20fps，偶尔会达到30ish。

When I add a Preview usecase to run along side it, and set enableTorch(true), the ImageAnalysis usecase will suddenly start getting more like 20fps, with it occasionally peaking up to 30ish.

很明显，预览用例会更改相机的某些自动曝光状态？

Clearly the Preview usecase alters some of the AutoExposure state of the camera?

这是我当前的偷窥设置...

Here's a snipper of my current setup...

 private fun startCameraAnalysis() {
        val metrics = DisplayMetrics().also { viewFinder.display.getRealMetrics(it) }
        var resolution = Size(metrics.widthPixels, metrics.heightPixels)
        resolution = Size(640, 480) //set to fixed size for testing

        val aspectRatio = Rational(resolution.width, resolution.height)
        val rotation = viewFinder.display.rotation

        // Setup image analysis pipeline
        val analyzerConfig = ImageAnalysisConfig.Builder().apply {
            val analyzerThread = HandlerThread(
                "LuminosityAnalysis").apply { start() }
            setCallbackHandler(Handler(analyzerThread.looper))
     setImageReaderMode(ImageAnalysis.ImageReaderMode.ACQUIRE_LATEST_IMAGE)
            setTargetRotation(rotation)
            setTargetAspectRatio(aspectRatio)
            setTargetResolution(resolution)
        }.build()

        // Setup preview pipeline
        val previewConfig = PreviewConfig.Builder().apply {
            setTargetRotation(rotation)
            setTargetAspectRatio(aspectRatio)
            setTargetResolution(resolution)
        }.build()

        // Build Preview useCase
        val preview = Preview(previewConfig)
        preview.enableTorch(true)

        // Build Analysis useCase
        val analyzer = ImageAnalysis(analyzerConfig)
        analyzer.analyzer = LuminosityAnalyzer()

        CameraX.bindToLifecycle(this, preview, analyzer )
        preview.enableTorch(true)
}

是否有必要在ImageAnalysis周围更改CameraX中的摄像机设置以获得更高的帧速率？

Is there anyway to alter the camera settings in CameraX around the ImageAnalysis to get higher frame rates?

实际上是否真的可以更改诸如传感器持续时间，ISO，曝光度之类的东西？

Is there anyway at all in fact to alter things like Sensor Duration, ISO, exposure?

推荐答案

所以我花了一些钱

事实证明，ImageAnalysisConfig不可扩展，因此您无法更改相机配置，而仅使用其中之一，则将使用默认的相机设置，我认为这会导致我的手机开启AE并达到16ish FPS。

It turns out, the ImageAnalysisConfig is not extendable, so you can't alter the camera configuration when just using one of those, so the default camera settings will be used which on my phone I think resulted in AE being on and hitting 16ish FPS.

如果您旋转设置一个PreviewConfig使其同时运行，然后可以使用Camera2Config.Extender对其进行扩展并直接更改camera2属性。这样可以提高相机预览的帧速率，分析器也将开始以相同的速率获取帧。

If you spin up a PreviewConfig to run along side it at the same time, you can then extend this with a Camera2Config.Extender and alter the camera2 properties directly. This can increase the camera preview frame rate, and the Analyser will also start getting frames at the same rate.

因此，例如，我将其添加到PreviewConfig中...

So for example, I add this to my PreviewConfig...

    // Create Camera2 extender
    var camera2Extender = Camera2Config.Extender(previewConfig)
        .setCaptureRequestOption(CaptureRequest.CONTROL_MODE, CaptureRequest.CONTROL_MODE_OFF)
        .setCaptureRequestOption(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF)
        .setCaptureRequestOption(CaptureRequest.CONTROL_AF_MODE, CaptureRequest.CONTROL_AF_MODE_OFF)
        .setCaptureRequestOption(CaptureRequest.CONTROL_AWB_MODE, CaptureRequest.CONTROL_AWB_MODE_OFF)
        .setCaptureRequestOption(CaptureRequest.FLASH_MODE, CaptureRequest.FLASH_MODE_TORCH)
        .setCaptureRequestOption(CaptureRequest.SENSOR_SENSITIVITY, 100)
        .setCaptureRequestOption(CaptureRequest.SENSOR_FRAME_DURATION, 16666666)
        .setCaptureRequestOption(CaptureRequest.SENSOR_EXPOSURE_TIME, 20400000)

因此，这在ImageAnalyser中开始达到30fps的罚款。

So this started hitting 30fps fine in the ImageAnalyser.

如果我想达到60，我可以设置...

If I want to hit 60, I can set...

.setCaptureRequestOption(CaptureRequest.CONTROL_AE_MODE, CaptureRequest.CONTROL_AE_MODE_OFF)
.setCaptureRequestOption(CaptureRequest.CONTROL_AE_TARGET_FPS_RANGE, Range(60,60))

显然是假设设备支持（60,60）的目标FPS范围。

Obviously assuming the device support (60,60) target FPS range.

因此，似乎在CameraX中仍然可以使用完整的Camera2逻辑，有点笨拙，它隐藏在Camera2Config扩展器中，仅适用于Preview用例。

So it seems the full Camera2 logic is still available in CameraX, its just a bit clunky that its a little hidden away in a Camera2Config extender, and this only works with Preview use cases.

这篇关于如何使用Android CameraX ImageAnalysis提高帧频？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！