使用 ArCore 时是否可以捕获高分辨率图像? [英] Is it possible to capture a High-Res image while using ArCore?

问题描述

在我的应用中，我试图在自定义相机视图中将 ArCore 用作某种相机助手".

In my app I'm trying to use ArCore as sort of a "camera assistant" in a custom camera view.

要清楚 - 我想在他的相机中为用户显示图像，并让他捕捉不包含 AR 模型的图像.

To be clear - I want to display images for the user in his camera and have him capture images that don't contain the AR models.

据我所知，为了使用 ArCore 捕获图像，我必须使用 Camera2 API，该 API 通过将会话配置为使用共享相机"来启用.

From what I understand, in order to capture an image with ArCore I'll have to use the Camera2 API which is enabled by configuring the session to use the "shared Camera".

但是，我似乎无法将相机配置为使用任何高端分辨率(我使用的是像素 3，所以我应该能够达到 12MP).

However, I can't seem to configure the camera to use any high-end resolutions (I'm using pixel 3 so I should be able to go as high as 12MP).

在共享相机示例"中，他们在 Camera2 和 ArCore 之间切换(遗憾的是没有用于 CameraX 的 API)并且存在几个问题:

In the "shared camera example", they toggle between Camera2 and ArCore (a shame there's no API for CameraX) and it has several problems:

1. 在 ArCore 模式下，图像模糊(我认为这是因为深度传感器在他们的文档中被禁用)
2. 在 Camera2 模式下，我根本无法提高分辨率.
3. 在显示来自 ArCore 的模型时，我无法使用 Camera2 API 捕获图像.

目前这个要求是否可行?

Is this requirement at all possible at the moment?

推荐答案

我还没有使用 ARCore 的共享摄像头，但我可以就你的问题的要点说几句.

I have not worked yet with shared camera with ARCore, but I can say a few things regarding the main point of your question.

在 ARCore 中，您可以配置 CPU 图像大小和 GPU 图像大小.您可以通过检查所有 可用的相机配置(可通过Session.getSupportedCameraConfigs(CameraConfigFilter cameraConfigFilter)获得)并通过将其传递回ARCore Session来选择您喜欢的相机配置.在每个 CameraConfig 上，您可以检查哪个 CPU您将获得的图像大小和 GPU 纹理大小.

In ARCore you can configure both CPU image size and GPU image size. You can do that by checking all available camera configurations (available through Session.getSupportedCameraConfigs(CameraConfigFilter cameraConfigFilter)) and selecting your preferred one by passing it back to the ARCore Session. On each CameraConfig you can check which CPU image size and GPU texture size you will get.

可能您目前正在使用(可能是默认情况下?)一个具有最低 CPU 图像的 CameraConfig，如果我没记错的话是 640x480 像素，所以是的，它在渲染时看起来确实很模糊(但在这方面与深度传感器无关).

Probably you are currently using (maybe by default?) a CameraConfig with the lowest CPU image, 640x480 pixels if I remember correctly, so yes it definitely looks blurry when rendered (but nothing to do with depth sensor in this regard).

听起来您可以选择更高的 CPU 映像，这样就可以了……但不幸的是，情况并非如此，因为该配置适用于每一帧.获得更高分辨率的 CPU 图像将导致性能低得多.当我测试这个时，我在我的测试设备上每秒获得大约 3-4 帧，绝对不理想.

Sounds like you could just select a higher CPU image and you're good to go... but unfortunately that's not the case because that configuration applies to every frame. Getting higher resolution CPU images will result in much lower performance. When I tested this I got about 3-4 frames per second on my test device, definitely not ideal.

那现在怎么办?我认为您有两个选择:

1. 暂停 ARCore 会话，切换到更高 CPU 图像 1 帧，获取图像并切换回正常"图像配置.
2. 可能您已经获得了不错的 GPU 图像，由于相机预览，可能不是最好的，但希望足够好?不确定你是如何渲染它的，但你可以使用一些 OpenGL 技能来复制该纹理.当然，不是直接的，因为整个 GL_TEXTURE_EXTERNAL_OES 事情......但是将它渲染到另一个帧缓冲区，然后读取附加到它的纹理可以工作.当然，您可能需要自己处理纹理坐标(完整图像与可见区域)，但这是另一个主题.

1. Pause the ARCore session, switch to a higher CPU image for 1 frame, get the image and switch back to the "normal" configuration.
2. Probably you are already getting a nice GPU image, maybe not the best due to camera Preview, but hopefully good enough? Not sure how you are rendering it, but with some OpenGL skills you can copy that texture. Not directly, of course, because of the whole GL_TEXTURE_EXTERNAL_OES thing... but rendering it onto another framebuffer and then reading the texture attached to it could work. Of course you might need to deal with texture coordinates yourself (full image vs visible area) but that's another topic.

关于 CameraX，请注意它包装了 Camera2 API 以提供一些相机用例，以便应用程序开发人员不必担心相机生命周期.据我了解，ARCore 不适合使用 CameraX，因为我认为他们需要完全控制相机.

Regarding CameraX, note that it is wrapping Camera2 API in order to provide some camera use cases so that app developers don't have to worry about the camera lifecycle. As I understand it would not be suitable for ARCore to use CameraX as I imagine they need full control of the camera.

希望对你有所帮助！

这篇关于使用 ArCore 时是否可以捕获高分辨率图像?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！