在iOS 11 ARKit中获取相机视野 [英] Get camera field of view in iOS 11 ARKit

问题描述

我正在使用ARKit的ARSCNView在iPad上实时显示来自相机的视频。我将ARSCNView对象设置为Xcode的增强现实应用程序模板。我想知道是否有办法获得相机的视野？

I'm using a ARSCNView from ARKit to display live a video feed from the camera on the iPad. I have the ARSCNView object setup exactly as Xcode's Augmented Reality App template. I was wondering if there is a way to get the field of view of the camera?

@IBOutlet var sceneView: ARSCNView!

func start() {
    sceneView.delegate = self
    sceneView.session.run(ARWorldTrackingConfiguration())
    // Retrieve camera FOV here
}

推荐答案

有几种方法可以到这里，并且可能是错误的开始要小心。

There are a couple of ways to go here, and a possible false start to beware of.

如果你'已经通过SceneKit（ ARSCNView ）使用ARKit，您可能会认为ARKit会自动更新SceneKit相机（视图的 pointOfView camera ）以匹配ARKit使用的投影变换。这是正确的。

If you're already working with ARKit via SceneKit (ARSCNView), you might assume that ARKit is automatically updating the SceneKit camera (the view's pointOfView's camera) to match the projection transform used by ARKit. This is correct.

然而，ARKit直接设置 SCNCamera 的 projectionTransform 。使用 SCNCamera 的几何属性时，如 zNear 和 zFar 和 fieldOfView ，SceneKit导出用于渲染的投影矩阵。但是如果你设置 projectionTransform 直接，没有数学可以恢复near / far和xFov / yFov值，因此相应的 SCNCamera 属性无效。也就是说， sceneView.pointOfView.camera.fieldOfView 和类似的API总是返回ARKit应用程序的伪造结果。

However, ARKit directly sets the SCNCamera's projectionTransform. When you work with geometric properties of SCNCamera like zNear and zFar and fieldOfView, SceneKit derives a projection matrix for use in rendering. But if you set projectionTransform directly, there's no math that can recover the near/far and xFov/yFov values, so the corresponding SCNCamera properties are invalid. That is, sceneView.pointOfView.camera.fieldOfView and similar APIs always return bogus results for an ARKit app.

那么，你能做些什么呢？请继续阅读...

So, what can you do instead? Read on...

AR会话不断推出 ARFrame 对象通过其委托，或者您可以请求 currentFrame 来自它。每个框架都有一个 ARCamera 附件描述了成像参数，其中一个是 projectionMatrix 依赖于视野。 （还有前面提到的SceneKit projectionTransform ，它是相同的矩阵。）

An AR session continually vends ARFrame objects through its delegate, or you can request the currentFrame from it. Each frame has an ARCamera attached that describes the imaging parameters, one of which is a projectionMatrix that's dependent on field of view. (There's also the aforementioned SceneKit projectionTransform, which is the same matrix.)

标准3D投影矩阵包括缩放基于垂直视野和纵横比的术语。具体来说，矩阵如下所示：

A standard 3D projection matrix includes scaling terms that are based on the vertical field of view and aspect ratio. Specifically, the matrix looks like this:

[ xScale    0     0    0  ]   xScale = yScale * aspectRatio (width/height)
[   0    yScale   0    0  ]   yScale = 1 / tan(yFov/2)
[   0       0    nf1  nf2 ]   nf1 and nf2 relate to near and far clip planes,
[   0       0     -1   0  ]     so aren't relevant to field of view

所以你应该可以获得 yFov 解决 yScale 等式：

So you should be able to get yFov by solving the yScale equation:

let projection = session.currentFrame!.camera.projectionMatrix
let yScale = projection[1,1]
let yFov = 2 * atan(1/yScale) // in radians
let yFovDegrees = yFov * 180/Float.pi

对于水平视野，你可以乘以宽高比（具体地说，宽高比）：

And for horizontal field of view, you can multiply by the aspect ratio (specifically, the width/height ratio):

let imageResolution = session.currentFrame!.camera.imageResolution
let xFov = yFov * Float(imageResolution.width / imageResolution.height)

注意：这里，水平和垂直是相对于相机图像，无论您的设备或AR视图UI如何定向，它都是横向的。

Note: Here, "horizontal" and "vertical" are with respect to the camera image, which is natively in landscape orientation regardless of how your device or AR view UI are oriented.

但仔细观察，您可能会注意到 xFov / yFov 此处（以及 imageResolution 的宽高比）不一定与您的设备屏幕（尤其是在iPhone X上）或您正在绘制AR内容的视图相匹配。那是因为您测量了摄像机图像的视觉角度 ，而不是应用程序AR视图的视角。别担心，这也有一个API ...

If you look closely, though, you might notice that the aspect ratio between xFov/yFov here (and the aspect ratio of imageResolution) don't necessarily match that of your device screen (especially on iPhone X) or the view you're drawing AR content in. That's because you've measured the FOV angles of the camera image, not those of your app's AR view. Don't worry, there's an API for that, too...

ARCamera 提供了两个用于获取投影矩阵的API。除了我们刚刚过去的那个，还有 projectionMatrix（ for：viewportSize：zNear：zFar：） ，将演示文稿考虑在内。如果你想匹配的不是相机的FOV，而是FOV如何 ARSCNView 或 ARSKView （或Unity或虚幻，可能？）渲染您的AR场景，使用它，传递设备方向并查看您的视图。然后做与上面相同的数学运算：

ARCamera offers two APIs for getting a projection matrix. Besides the one we just went over, there's also projectionMatrix(for:viewportSize:zNear:zFar:), which takes presentation into account. If you want to match not the FOV of the camera, but the FOV of how ARSCNView or ARSKView (or Unity or Unreal, probably?) render your AR scene, use this, passing the device orientation and see of your view. Then do all the same math as above:

let imageResolution = session.currentFrame!.camera.imageResolution
let viewSize = sceneView.bounds.size
let projection = session.currentFrame!.camera.projectionMatrix(for: .portrait,
    viewportSize: viewSize, zNear: zNear, zFar: zFar)
let yScale = projection[1,1] // = 1/tan(fovy/2)
let yFovDegrees = 2 * atan(1/yScale) * 180/Float.pi
let xFovDegrees = yFovDegrees * Float(viewSize.height / viewSize.width)

你传递的内容 zNear 和 zFar 无关紧要，因为我们没有使用依赖于它的矩阵部分。 （您可能仍需要确保 zNear< zFar 和 zNear！= zFar！= 0 。）

What you pass for zNear and zFar doesn't matter, since we're not using the parts of the matrix that depend on that. (You might still need to ensure zNear < zFar and zNear != zFar != 0.)

注意：现在高度/宽度基于您的视图（或者更确切地说，是您传递的视图的属性）到 projectionMatrix（for：...））。在此示例中， yFov 相对于UI是垂直的，因为方向是 portrait ，因此您乘以高度/宽高比要获得 xFov 。如果你在风景中，你会乘以宽度/高度。

Note: Now the height/width are based on your view (or rather, the attributes of your view that you pass to projectionMatrix(for:...)). In this example, yFov is vertical with respect to the UI because the orientation is portrait, so you multiply by the height/width aspect ratio to get xFov. If you're in landscape, you multiply by width/height instead.

Camera Intrinsics

敏锐的观察者可能已经注意到上述计算忽略了投影矩阵的一部分。这是因为 FOV角度的定义是相机的光学属性，与此无关3D投影，因此整个投影矩阵是您可能不需要的中间结果。

Camera Intrinsics

Keen observers may have noticed that the above calculations ignore parts of the projection matrix. That's because the definition of FOV angle is an optical property of the camera, not anything to do with 3D projection, so a whole projection matrix is an intermediate result you might not really need.

ARCamera 也暴露了 intrinsics 描述相机光学特性的矩阵。此矩阵中对角线的第一个和第二个值是水平和垂直的焦距相机图像中的单个像素。如果您有焦距和图像宽度/高度，则可以根据定义计算FOV FOV角度：

ARCamera also exposes an intrinsics matrix that describes optical properties of the camera. The first and second values along the diagonal in this matrix are the horizontal and vertical focal length of a single pixel in the camera image. If you have focal length and image width/height, you can compute FOV per the definition of FOV angle:

let imageResolution = session.currentFrame!.camera.imageResolution
let intrinsics = session.currentFrame!.camera.intrinsics
let xFovDegrees = 2 * atan(Float(imageResolution.width)/(2 * intrinsics[0,0])) * 180/Float.pi
let yFovDegrees = 2 * atan(Float(imageResolution.height)/(2 * intrinsics[1,1])) * 180/Float.pi

注意：与使用 projectionMatrix 的版本一样，这取决于尺寸和摄像机图像的始终横向方向，而不是设备屏幕或您正在显示AR内容的视图。如果您需要基于视口的内容，请向上滚动回使用Viewport的投影矩阵 。

Note: Like the version that uses projectionMatrix, this is based on the size and always-landscape orientation of the camera image, not the device screen or the view you're displaying AR content in. If you need something based on the viewport instead, scroll back up to "Projection Matrix with Viewport".

这篇关于在iOS 11 ARKit中获取相机视野的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！