Android Camera2 捕获图像倾斜 [英] Android Camera2 capture image skewed

问题描述

更新:这看起来与此有关:来自 Android camera2 API 的图像数据翻转 &在 Galaxy S5 上压扁 - 我认为这是一个错误，因为 Nexus 5/6 工作正常，需要获得完整的传感器尺寸然后手动裁剪以达到所需的纵横比是没有意义的，也可能不使用支持的"输出尺寸也是如此！

Update: This looks like it's related to this: Image data from Android camera2 API flipped & squished on Galaxy S5 - I consider this as a bug since Nexus 5/6 works correctly and it makes no sense to need to obtain full sensor size and then cropping manually to reach the desired aspect ratio, might as well not using "supported" output sizes as well!

问题:

1. 使用 Camera2 API 获取相机的特征，并提取适合 MediaCodec.class 的输出尺寸
2. 使用合适的相机输出尺寸之一创建 MediaCodec 输入表面.将输出提供给某些 MediaMuxer 或其他任何东西，以查看输出.
3. 使用编解码器创建的表面作为目标启动相机捕获请求.

4. 编解码器输出具有正确的大小.但结果因设备而异:

1. Get characteristics of a camera using Camera2 API, and extract output sizes suitable for a MediaCodec.class
2. Create a MediaCodec input surface with one of the suitable camera output sizes. Feed the output to some MediaMuxer or whatever, to see the output.
3. Start camera capture requests using the codec's created surface as the target.

4. Codec output has the correct size. But the result differs by device:

• Nexus 5/6:在 Android 5/6 上一切正常.
• Android 5.1 的三星平板电脑:对于某些分辨率，图像明显被拉伸，表明相机输出分辨率与表面尺寸不匹配.开始旋转相机时变得非常明显 - 图像变得越来越歪斜，因为它没有与 X/Y 轴对齐.对于其他一些分辨率，输出正常.这里没有与大小或纵横比相关的模式.

没问题，有人会说.也许表面没有完全按照指定的宽度和高度创建，或者其他任何东西(即使输出大小是专门为 MediaCodec.class 目标提取的).

No problem, one would say. Maybe the surface is not created exactly at the specified width and height, or whatever (even if the output sizes were extracted specifically for a MediaCodec.class target).

因此，我创建了一个 OpenGL 上下文，为其生成了一个纹理和 SurfaceTexture，将其默认缓冲区大小设置为相机输出大小，并使用该纹理创建了一个 Surface.我不会详细介绍将其绘制到 TextureView 或返回 MediaCodec 的 EGL 表面的细节.结果是相同的 - Camera2 捕获请求仅针对某些分辨率输出失真图像.深入挖掘:在 updateTexImage 之后立即在 SurfaceTexture 上调用 getTransformMatrix - 正如预期的那样，矩阵始终是单位矩阵.

So, I created an OpenGL context, generated a texture, a SurfaceTexture for it, set its default buffer size to the camera output size, and created a Surface using the texture. I won't go into the gory details of drawing that to a TextureView or back to the MediaCodec's EGL surface. The result is the same - the Camera2 capture requests outputs a distorted image only for some resolutions. Digging deeper: calling getTransformMatrix on the SurfaceTexture immediately after updateTexImage - the matrix is always the identity matrix, as expected.

所以，这里真正的问题是相机没有以提供的目标表面的大小进行捕捉.因此，解决方案是获取相机正在捕获的实际尺寸，其余的是纯 GL 矩阵变换以正确绘制.但是 - 我如何得到它?

So, the real problem here is that the camera is NOT capturing at the size of the provided target surface. The solution would thereby be to get the actual size the camera is capturing, and the rest is pure GL matrix transforms to draw correctly. But - HOW DO I GET THAT?

注意:使用旧的相机 API，与目标(MediaCodec 的或自定义的)完全相同的预览尺寸"和相同表面 - 一切都很好！但是我不能使用旧的相机 API，因为它既已弃用，而且最大捕获尺寸似乎也为 1080p，而 Camera2 API 超出了这个范围，我需要支持 4k 录制.

Note: using the old Camera API, with exactly the same "preview size" and the same surface as the target (either MediaCodec's or the custom one) - ALL IS FINE! But I can't use the old camera API, since it's both deprecated and also seems to have a max capture size of 1080p, while the Camera2 API goes beyond that, and I need to support 4k recording.

推荐答案

是的，这是三星设备上的一个错误.

So yes, this is a bug on those Samsung devices.

通常，当您在输出时要求多个不同的纵横比时，会发生这种情况，并且特定于设备的相机代码在正确裁剪和缩放所有这些时会自行跳闸.您可以通过确保所有请求的尺寸具有相同的纵横比来避免这种情况.

Generally this happens when you ask for multiple different aspect ratios on output, and the device-specific camera code trips over itself on cropping and scaling all of them correctly. You may be able to avoid it by ensuring all requested sizes have the same aspect ratio.

分辨率可能实际上是您所要求的 - 但它被错误地缩放(您可以使用有问题的大小的 ImageReader 进行测试，在那里您可以获得一个可以戳的显式缓冲区.)

The resolution is probably actually what you asked for - but it's been incorrectly scaled (you could test this with an ImageReader at the problematic size, where you get an explicit buffer you can poke at.)

我们正在向 Android 合规性测试中添加额外的测试，以确保不会继续发生此类延伸输出.

We are adding additional testing to the Android compliance tests to try to ensure these kinds of stretched outputs don't continue to happen.

这篇关于Android Camera2 捕获图像倾斜的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！