可以使用 CATransform3D 获取 Face Mesh 中的眼睛大小尺寸吗? [英] Can CATransform3D be used to get eye size dimensions in Face Mesh?

问题描述

我正在尝试使用 ARKit 的 3D Face Mesh 获取眼睛的宽度和 2 只眼睛的距离.

I am trying to get width of the eyes and distance of 2 eyes using 3D Face Mesh of ARKit.

我用过ARAnchor的CATransform3D；

 struct CATransform3D
{
  CGFloat m11, m12, m13, m14;
  CGFloat m21, m22, m23, m24;
  CGFloat m31, m32, m33, m34;
  CGFloat m41, m42, m43, m44;
};

下面是我的代码；

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {

guard let faceAnchor = anchor as? ARFaceAnchor else { return }

let leftcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkLeft]?.caTransform3DValue)!

let rightcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkRight]?.caTransform3DValue)!

print("  m11 : \(String(describing:leftcaTransform3DValue.m11)) m12 : \(String(describing:leftcaTransform3DValue.m12)) m13 : \(String(describing:leftcaTransform3DValue.m13)) m14 : \(String(describing:leftcaTransform3DValue.m14)) m21 : \(String(describing:leftcaTransform3DValue.m21)) m22 : \(String(describing:leftcaTransform3DValue.m22)) m23 : \(String(describing:leftcaTransform3DValue.m23)) m24 : \(String(describing:leftcaTransform3DValue.m24)) m31 : \(String(describing:leftcaTransform3DValue.m31)) m32 : \(String(describing:leftcaTransform3DValue.m32)) m33 : \(String(describing:leftcaTransform3DValue.m33)) m34 : \(String(describing:leftcaTransform3DValue.m34)) m41 : \(String(describing:leftcaTransform3DValue.m41)) m42 : \(String(describing:leftcaTransform3DValue.m42)) m43 : \(String(describing:leftcaTransform3DValue.m43)) m44 : \(String(describing:leftcaTransform3DValue.m44)) " )
}

作为 leftcaTransform3DValue 的结果，我得到了类似的值；

And as a result of leftcaTransform3DValue I got values like;

m11 =  -5.22553711590422e-315
...
...
...
m44 =   2.13285635582599e-314

与 rightcaTransform3DValue 相同.

所以我的问题是这些值是否指定了任何尺寸或尺寸测量?

So my question is do these values specify any dimension or size measurement?

我可以计算眼睛的宽度和两只眼睛之间的距离吗?

Can I calculate width of the eyes and distance between two eyes?

非常感谢任何帮助.

推荐答案

所以我的问题是这些值是否指定了任何尺寸或尺寸测量?

So my question is do these values specify any dimension or size measurement?

没有.你得到的数字是无稽之谈，因为你得到它们的方式是......也许不是完全无稽之谈，但非常接近.

No. The numbers you’re getting are nonsense, because the way you’re getting them is... maybe not quite nonsense, but pretty close.

blendShapes 字典ARFaceAnchor 被记录为具有 NSNumber 类型的值，其中 NSNumber 的基础数值是介于 0.0 和 1.0 之间的浮点数.

The blendShapes dictionary on ARFaceAnchor is documented as having values of type NSNumber, where the underlying numeric value of the NSNumber is a Float between 0.0 and 1.0.

NSNumber 是许多可能的对象包装器标量数值的类型.它有一些方法可以将其底层值获取为各种类型(通过转换为相同数字的不同表示).但鉴于这些特定数字被记录为 0 到 1 之间的浮点值，获取 intValue 或 boolValue 等没有多大意义.

NSNumber is an object wrapper for many possible types of scalar numeric values. It has methods for fetching its underlying value as various types (by converting to a different representation of the same number). But given that these particular numbers are documented to be floating-point values between 0 and 1, there’s not much sense to fetching the intValue or boolValue, etc.

NSNumber 是 NSValue，它是许多种类的对象包装器，这些类型不能以其他方式表达为对象——范围、大小、指针和 3D 变换矩阵等.这些类型不能像数字那样相互转换，所以从 NSValue 中获取唯一有意义的类型是它创建时使用的类型.任何其他类型都会让你胡说八道.

NSNumber is a subclass of NSValue, which is an object wrapper for many kinds of types not otherwise expressible as objects — ranges, sizes, pointers, and 3D transform matrices, among others. Those types can’t be converted between each other like numbers can, so the only type that’s meaningful to fetch out of an NSValue is the type it was created with. Any other type gives you nonsense.

回到blendShapes —它进一步记录了字典中的每个混合形状值不仅仅是一个数字，而是一个告诉您动画参数进度的数字.eyeBlinkLeft 未声明告诉您有关左眼的位置或大小的任何信息 - 它会告诉您左眼睑的眨眼"(闭合)程度.

Back to blendShapes — it’s further documented that each blend shape value in the dictionary isn’t just a number, but a number that tells you the progress of an animation parameter. eyeBlinkLeft doesn’t claim to tell you anything about where or how big the left eye is — it tells you how "blinked" (closed) the left eyelid is.

你找错了树，但如果你查看你正在使用的类和属性的文档，你将能够更好地做出有根据的猜测.

You’re barking up the wrong tree, but if you look at the docs for the classes and properties you’re using, you’ll be better able to make educated guesses later.

我可以计算眼睛的宽度和两只眼睛之间的距离吗?

Can I calculate width of the eyes and distance between two eyes?

更新:在ARKit 2"(iOS 12 中的 ARKit)中，leftEyeTransform 和 rightEyeTransform 提供每个眼球中心的 3D 位置(相对于面部锚点).(还有每只眼睛的方向.)这可能对您的用例有所帮助，但如果您真正想要的是与瞳孔的位置/大小或眼睛张开有关...

Update: In "ARKit 2", aka ARKit in iOS 12, the leftEyeTransform and rightEyeTransform provide the 3D position (relative to the face anchor) of the center of each eyeball. (Also the orientation of each eye.) That might help your use case, but if what you're actually after has to do with the position/size of the pupils or the eye openings...

没有 API 可以为您执行此操作.ARKit 确实提供了您可以用来自己查找的信息，但不能保证始终有效.

There’s no API that’ll do this for you. ARKit does provide information you could use to find it out yourself, but not in a way that’s guaranteed to always work.

ARFaceGeometry 为您提供一个三角形网格映射脸上有几百个左右的点，在一个会话中拓扑稳定.也就是说，例如，假设网格中的第 57 个顶点是鼻尖，即使面部皱纹和拉伸以及该点相对于其他点的位置发生变化，它也会留在鼻尖上.

ARFaceGeometry gives you a triangle mesh that maps a few hundred or so points on the face, in a way that’s topologically stable within a session. That is, for example, assuming the 57th vertex in the mesh is the tip of the nose, it’ll stay on the tip of the nose even as the face wrinkles and stretches and that point’s position relative to other points changes.

问题:

• API 不会告诉您哪些顶点(网格中的点)是哪些(就眼角、鼻尖等面部标志而言).
• 网格的拓扑结构在会话内是稳定的，但 Apple 不保证它不会在 iOS 版本、设备等之间发生变化.

因此，通过一些实验，您可能能够确定哪些顶点是左眼的内角、左眼的外角等.一旦您这样做，您就可以查看它们的位置来估计有用的诸如眼睛宽度、瞳孔间距等量.但是，这些测量基于对可能并不总是保持的网格的假设，因此您不知道应用程序用户何时会中断.

So, though some experimentation you might be able to work out which vertices are the inside corner of the left eye, the outside corner of the left eye, etc. Once you do that, you can look at their positions to estimate useful quantities like eye width, inter-pupillary distance, etc. However, those measurements are based on assumptions about the mesh that might not always hold, so you don’t know when it’ll break for users of your app.

这篇关于可以使用 CATransform3D 获取 Face Mesh 中的眼睛大小尺寸吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！