ARKit中的FaceTracking –如何显示"lookAtPoint"屏幕上 [英] FaceTracking in ARKit – How to display the "lookAtPoint" on the screen

问题描述

ARKit的ARFaceTrackingConfiguration将ARFaceAnchor与有关脸部的位置和方向的信息一起放置到场景上.除其他外，此锚具有我感兴趣的lookAtPoint属性.我知道此向量相对于面部.如何在屏幕上为此位置绘制一个点，这意味着我该如何转换该点的坐标?

The ARFaceTrackingConfiguration of ARKit places ARFaceAnchor with information about the position and orientation of the face onto the scene. Among others, this anchor has the lookAtPoint property that I'm interested in. I know that this vector is relative to the face. How can I draw a point on the screen for this position, meaning how can I translate this point's coordinates?

推荐答案

.lookAtPoint 实例属性仅用于方向估计.

.lookAtPoint instance property is for direction's estimation only.

Apple文档说:.lookAtPoint是人脸坐标空间中的一个位置，仅估计人脸方向的视线.它是三个标量值的向量，它只是可获取的，而不是可设置的:

Apple documentation says: .lookAtPoint is a position in face coordinate space that is estimating only the gaze of face's direction. It's a vector of three scalar values, and it's just gettable, not settable:

var lookAtPoint: SIMD3<Float> { get }

换句话说，这是两个量乘积的结果向量– .rightEyeTransform和.leftEyeTransform实例属性(也只是可获取的):

In other words, this is the resulting vector from the product of two quantities – .rightEyeTransform and .leftEyeTransform instance properties (which also are just gettable):

var rightEyeTransform: simd_float4x4 { get }
var leftEyeTransform: simd_float4x4 { get }

关于如何使用此实例属性，这是一个假想的情况:

Here's an imaginary situation on how you could use this instance property:

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {

    if let faceAnchor = anchor as? ARFaceAnchor,
       let faceGeometry = node.geometry as? ARSCNFaceGeometry {

        if (faceAnchor.lookAtPoint.x >= 0) {         // Looking (+X)
            faceGeometry.firstMaterial?.diffuse.contents = UIImage(named: "redTexture.png")
        } else {                                     // Looking (-X)
            faceGeometry.firstMaterial?.diffuse.contents = UIImage(named: "cyanTexture.png")
        }

        faceGeometry.update(from: faceAnchor.geometry)
        facialExrpession(anchor: faceAnchor)

        DispatchQueue.main.async {
            self.label.text = self.textBoard
        }
    }
}

这是显示ARFaceTrackingConfiguration()的轴方向的图像:

And here's an image showing axis directions for ARFaceTrackingConfiguration():

回答您的问题:

我可以说您不能直接管理该点的坐标，因为它是仅可获取的属性(并且只有XYZ方向，而不是XYZ平移).

I could say that you can't manage this point's coordinates directly because it's gettable-only property (and there is just XYZ orientation, not XYZ translation).

因此，如果同时需要平移和旋转，请改用.rightEyeTransform和.lefttEyeTransform实例属性.

So if you need both – translation and rotation – use .rightEyeTransform and .lefttEyeTransform instance properties instead.

有两种投影方法:

第一.在SceneKit/ARKit中，您需要采用以下实例方法用于将点投影到2D视图上(对于sceneView实例):

FIRST. In SceneKit/ARKit you need to take the following instance method for projecting a point onto 2D view (for sceneView instance):

func projectPoint(_ point: SCNVector3) -> SCNVector3

或:

let sceneView = ARSCNView()
sceneView.projectPoint(myPoint)

第二.在ARKit中，您需要采用以下实例方法来投影指向2D视图(对于arCamera实例):

SECOND. In ARKit you need to take the following instance method for projecting a point onto 2D view (for arCamera instance):

func projectPoint(_ point: simd_float3, 
              orientation: UIInterfaceOrientation, 
             viewportSize: CGSize) -> CGPoint

或:

let camera = ARCamera()
camera.projectPoint(myPoint, orientation: myOrientation, viewportSize: vpSize)

此方法可帮助您将一个点从场景的3D世界坐标系投影到渲染器的2D像素坐标系.

This method helps you project a point from the 3D world coordinate system of the scene to the 2D pixel coordinate system of the renderer.

还有相反的方法(用于取消投影点):

There's also the opposite method (for unprojecting a point):

func unprojectPoint(_ point: SCNVector3) -> SCNVector3

...和ARKit相反的方法(用于取消投影点):

...and ARKit's opposite method (for unprojecting a point):

@nonobjc func unprojectPoint(_ point: CGPoint, 
            ontoPlane planeTransform: simd_float4x4, 
                         orientation: UIInterfaceOrientation, 
                        viewportSize: CGSize) -> simd_float3?

这篇关于ARKit中的FaceTracking –如何显示"lookAtPoint"屏幕上的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！