在 iOS 11 ARKit 中获取相机视野 [英] Get camera field of view in iOS 11 ARKit

问题描述

我正在使用 ARKit 的 ARSCNView 来显示来自 iPad 上摄像头的实时视频源.我的 ARSCNView 对象设置与 Xcode 的增强现实应用程序模板完全相同.我想知道是否有办法获得相机的视野?

@IBOutlet var sceneView: ARSCNView！功能开始(){场景视图.delegate = selfsceneView.session.run(ARWorldTrackingConfiguration())//在此处检索相机 FOV}

解决方案

这里有几种方法，但要注意可能的错误开始.

⚠️ ARKit + SceneKit(不正确)

如果您已经通过 SceneKit (ARSCNView) 使用 ARKit，您可能会认为 ARKit 会自动更新 SceneKit 相机(视图的 pointOfView 的 camera) 以匹配 ARKit 使用的投影变换.这是正确的.

但是ARKit直接设置了SCNCamera的projectionTransform.当您使用 SCNCamera 的几何属性时，例如 zNear 和 zFar 和 fieldOfView，SceneKit 派生出一个投影矩阵用于渲染.但是如果你直接设置 projectionTransform没有数学可以恢复近/远和 xFov/yFov 值，因此相应的 SCNCamera 属性无效.也就是说，sceneView.pointOfView.camera.fieldOfView 和类似的 API 总是为 ARKit 应用返回虚假结果.

那么，你能做些什么呢?继续阅读...

投影矩阵

AR 会话通过其不断地出售 ARFrame 对象委托，或者您可以请求 currentFrame从中.每个框架都有一个 ARCamera 附加描述成像参数，其中之一是依赖的 projectionMatrix在视野范围内.(还有前面提到的 SceneKit projectionTransform，这是同一个矩阵.)

标准 3D 投影矩阵包括基于垂直视野和纵横比的缩放项.具体来说，矩阵如下所示:

[ xScale 0 0 0 ] xScale = yScale * aspectRatio (width/height)[ 0 yScale 0 0 ] yScale = 1/tan(yFov/2)[ 0 0 nf1 nf2 ] nf1 和 nf2 与近剪裁平面和远剪裁平面有关，[ 0 0 -1 0 ] 所以与视野无关

所以你应该能够通过求解yScale方程得到yFov:

let projection = session.currentFrame!.camera.projectionMatrix让 yScale = 投影[1,1]让 yFov = 2 * atan(1/yScale)//弧度让 yFovDegrees = yFov * 180/Float.pi

对于水平视野，您可以乘以纵横比(具体来说，宽/高比):

让 imageResolution = session.currentFrame!.camera.imageResolution让 xFov = yFov * Float(imageResolution.width/imageResolution.height)

<块引用>

注意:此处，水平"和垂直"是相对于相机图像而言的，无论您的设备或 AR 视图 UI 的方向如何，相机图像本身都是横向的.

不过，如果您仔细观察，您可能会注意到此处 xFov/yFov 之间的纵横比(以及 imageResolution 的纵横比)) 不一定与您的设备屏幕(尤其是在 iPhone X 上)或您绘制 AR 内容的视图相匹配.那是因为您测量了相机图像的 FOV 角度，而不是应用程序的 AR 视图中的那些.别担心，我们也有一个 API...

带视口的投影矩阵

ARCamera 提供了两个用于获取投影矩阵的 API.除了我们刚刚浏览的那个，还有 projectionMatrix(for:viewportSize:zNear:zFar:)，它考虑了演示.如果您想匹配的不是相机的 FOV，而是 ARSCNView 或 ARSKView(或 Unity 或 Unreal，可能?)渲染 AR 场景的 FOV，请使用此，传递设备方向并查看您的视图.然后做与上面相同的所有数学运算:

让 imageResolution = session.currentFrame!.camera.imageResolution让 viewSize = sceneView.bounds.size让投影 = session.currentFrame!.camera.projectionMatrix(for: .portrait,viewportSize: viewSize, zNear: zNear, zFar: zFar)让 yScale = 投影[1,1]//= 1/tan(fovy/2)让 yFovDegrees = 2 * atan(1/yScale) * 180/Float.pi让 xFovDegrees = yFovDegrees * Float(viewSize.height/viewSize.width)

你为 zNear 和 zFar 传递什么并不重要，因为我们没有使用依赖于它的矩阵部分.(您可能仍需要确保 zNear 和 zNear != zFar != 0.)

<块引用>

注意:现在高度/宽度基于您的视图(或者更确切地说，是您传递给 projectionMatrix(for:...) 的视图的属性)).在这个例子中，yFov 相对于 UI 是垂直的，因为方向是 portrait，所以你乘以高度/宽度纵横比得到 xFov.如果您处于横向，则改为乘以宽度/高度.

相机内在函数

敏锐的观察者可能已经注意到上述计算忽略了投影矩阵的一部分.这是因为 FOV 角度的定义是相机的光学属性，与 3D 无关投影，因此整个投影矩阵是您可能并不真正需要的中间结果.

ARCamera 还公开了一个 intrinsics 描述相机光学特性的矩阵.此矩阵中沿对角线的第一个和第二个值是单个的水平和垂直焦距相机图像中的像素.如果您有焦距和图像宽度/高度，则可以根据 定义计算 FOV视场角:

让 imageResolution = session.currentFrame!.camera.imageResolution让内在函数 = session.currentFrame!.camera.intrinsics让 xFovDegrees = 2 * atan(Float(imageResolution.width)/(2 * internals[0,0])) * 180/Float.pi让 yFovDegrees = 2 * atan(Float(imageResolution.height)/(2 * internals[1,1])) * 180/Float.pi

<块引用>

注意:与使用 projectionMatrix 的版本一样，这是基于相机图像的大小和始终横向方向，而不是设备屏幕或显示 AR 内容的视图.如果您需要基于视口的内容，请向上滚动到带视口的投影矩阵".

I'm using a ARSCNView from ARKit to display live a video feed from the camera on the iPad. I have the ARSCNView object setup exactly as Xcode's Augmented Reality App template. I was wondering if there is a way to get the field of view of the camera?

@IBOutlet var sceneView: ARSCNView!

func start() {
    sceneView.delegate = self
    sceneView.session.run(ARWorldTrackingConfiguration())
    // Retrieve camera FOV here
}

解决方案

There are a couple of ways to go here, and a possible false start to beware of.

⚠️ ARKit + SceneKit (incorrect)

If you're already working with ARKit via SceneKit (ARSCNView), you might assume that ARKit is automatically updating the SceneKit camera (the view's pointOfView's camera) to match the projection transform used by ARKit. This is correct.

However, ARKit directly sets the SCNCamera's projectionTransform. When you work with geometric properties of SCNCamera like zNear and zFar and fieldOfView, SceneKit derives a projection matrix for use in rendering. But if you set projectionTransform directly, there's no math that can recover the near/far and xFov/yFov values, so the corresponding SCNCamera properties are invalid. That is, sceneView.pointOfView.camera.fieldOfView and similar APIs always return bogus results for an ARKit app.

So, what can you do instead? Read on...

Projection Matrix

An AR session continually vends ARFrame objects through its delegate, or you can request the currentFrame from it. Each frame has an ARCamera attached that describes the imaging parameters, one of which is a projectionMatrix that's dependent on field of view. (There's also the aforementioned SceneKit projectionTransform, which is the same matrix.)

A standard 3D projection matrix includes scaling terms that are based on the vertical field of view and aspect ratio. Specifically, the matrix looks like this:

[ xScale    0     0    0  ]   xScale = yScale * aspectRatio (width/height)
[   0    yScale   0    0  ]   yScale = 1 / tan(yFov/2)
[   0       0    nf1  nf2 ]   nf1 and nf2 relate to near and far clip planes,
[   0       0     -1   0  ]     so aren't relevant to field of view

So you should be able to get yFov by solving the yScale equation:

let projection = session.currentFrame!.camera.projectionMatrix
let yScale = projection[1,1]
let yFov = 2 * atan(1/yScale) // in radians
let yFovDegrees = yFov * 180/Float.pi

And for horizontal field of view, you can multiply by the aspect ratio (specifically, the width/height ratio):

let imageResolution = session.currentFrame!.camera.imageResolution
let xFov = yFov * Float(imageResolution.width / imageResolution.height)

Note: Here, "horizontal" and "vertical" are with respect to the camera image, which is natively in landscape orientation regardless of how your device or AR view UI are oriented.

If you look closely, though, you might notice that the aspect ratio between xFov/yFov here (and the aspect ratio of imageResolution) don't necessarily match that of your device screen (especially on iPhone X) or the view you're drawing AR content in. That's because you've measured the FOV angles of the camera image, not those of your app's AR view. Don't worry, there's an API for that, too...

Projection Matrix with Viewport

ARCamera offers two APIs for getting a projection matrix. Besides the one we just went over, there's also projectionMatrix(for:viewportSize:zNear:zFar:), which takes presentation into account. If you want to match not the FOV of the camera, but the FOV of how ARSCNView or ARSKView (or Unity or Unreal, probably?) render your AR scene, use this, passing the device orientation and see of your view. Then do all the same math as above:

let imageResolution = session.currentFrame!.camera.imageResolution
let viewSize = sceneView.bounds.size
let projection = session.currentFrame!.camera.projectionMatrix(for: .portrait,
    viewportSize: viewSize, zNear: zNear, zFar: zFar)
let yScale = projection[1,1] // = 1/tan(fovy/2)
let yFovDegrees = 2 * atan(1/yScale) * 180/Float.pi
let xFovDegrees = yFovDegrees * Float(viewSize.height / viewSize.width)

What you pass for zNear and zFar doesn't matter, since we're not using the parts of the matrix that depend on that. (You might still need to ensure zNear < zFar and zNear != zFar != 0.)

Note: Now the height/width are based on your view (or rather, the attributes of your view that you pass to projectionMatrix(for:...)). In this example, yFov is vertical with respect to the UI because the orientation is portrait, so you multiply by the height/width aspect ratio to get xFov. If you're in landscape, you multiply by width/height instead.

Camera Intrinsics

Keen observers may have noticed that the above calculations ignore parts of the projection matrix. That's because the definition of FOV angle is an optical property of the camera, not anything to do with 3D projection, so a whole projection matrix is an intermediate result you might not really need.

ARCamera also exposes an intrinsics matrix that describes optical properties of the camera. The first and second values along the diagonal in this matrix are the horizontal and vertical focal length of a single pixel in the camera image. If you have focal length and image width/height, you can compute FOV per the definition of FOV angle:

let imageResolution = session.currentFrame!.camera.imageResolution
let intrinsics = session.currentFrame!.camera.intrinsics
let xFovDegrees = 2 * atan(Float(imageResolution.width)/(2 * intrinsics[0,0])) * 180/Float.pi
let yFovDegrees = 2 * atan(Float(imageResolution.height)/(2 * intrinsics[1,1])) * 180/Float.pi

Note: Like the version that uses projectionMatrix, this is based on the size and always-landscape orientation of the camera image, not the device screen or the view you're displaying AR content in. If you need something based on the viewport instead, scroll back up to "Projection Matrix with Viewport".

这篇关于在 iOS 11 ARKit 中获取相机视野的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！