可以使用CATransform3D获取Face Mesh中的眼睛大小尺寸吗? [英] Can CATransform3D be used to get eye size dimensions in Face Mesh?

问题描述

我正在尝试使用ARKit的3D Face Mesh获得眼睛的宽度和2眼的距离.

我使用了 ARAnchor 的 CATransform3D ；

 struct CATransform3D
{
  CGFloat m11, m12, m13, m14;
  CGFloat m21, m22, m23, m24;
  CGFloat m31, m32, m33, m34;
  CGFloat m41, m42, m43, m44;
};

下面是我的代码；

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {

guard let faceAnchor = anchor as? ARFaceAnchor else { return }

let leftcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkLeft]?.caTransform3DValue)!

let rightcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkRight]?.caTransform3DValue)!

print("  m11 : \(String(describing:leftcaTransform3DValue.m11)) m12 : \(String(describing:leftcaTransform3DValue.m12)) m13 : \(String(describing:leftcaTransform3DValue.m13)) m14 : \(String(describing:leftcaTransform3DValue.m14)) m21 : \(String(describing:leftcaTransform3DValue.m21)) m22 : \(String(describing:leftcaTransform3DValue.m22)) m23 : \(String(describing:leftcaTransform3DValue.m23)) m24 : \(String(describing:leftcaTransform3DValue.m24)) m31 : \(String(describing:leftcaTransform3DValue.m31)) m32 : \(String(describing:leftcaTransform3DValue.m32)) m33 : \(String(describing:leftcaTransform3DValue.m33)) m34 : \(String(describing:leftcaTransform3DValue.m34)) m41 : \(String(describing:leftcaTransform3DValue.m41)) m42 : \(String(describing:leftcaTransform3DValue.m42)) m43 : \(String(describing:leftcaTransform3DValue.m43)) m44 : \(String(describing:leftcaTransform3DValue.m44)) " )
}

由于 leftcaTransform3DValue 的结果，我得到了类似的值；

m11 =  -5.22553711590422e-315
...
...
...
m44 =   2.13285635582599e-314

与 rightcaTransform3DValue 相同.

所以我的问题是这些值是否指定任何尺寸或尺寸度量?

我可以计算出眼睛的宽度和两只眼睛之间的距离吗?

我们非常感谢您的帮助.

解决方案

所以我的问题是这些值是否指定任何尺寸或尺寸度量?

不.您得到的数字是胡说八道，因为您获得数字的方式是……也许不是胡说八道，但很接近.

记录了ARFaceAnchor上的 blendShapes 词典例如具有类型NSNumber的值，其中NSNumber的基础数字值是介于0.0和1.0之间的浮点数.

NSNumber 是许多标量数值类型的对象包装.它具有多种方法来获取其基础值(通过转换为相同数字的不同表示形式).但是，鉴于这些特殊数字记录为0到1之间的浮点值，因此获取intValue或boolValue等

并没有多大意义.

NSNumber是 NSValue 的子类.许多类型的种类的包装，否则不能表示为对象-范围，大小，指针和3D变换矩阵等.这些类型无法像数字一样在彼此之间转换，因此唯一可从NSValue中提取的类型是使用它创建的类型.任何其他类型都会给您带来胡扯.

返回 blendShapes –进一步记录了每个词典中的blend shape值不仅是数字，而且是一个数字，它告诉您动画参数的进度. eyeBlinkLeft 并不要求告诉您任何有关左眼的位置或大小-告诉您左眼皮眨眼"(闭合)的程度.

您正在树错树丛，但是如果您查看正在使用的类和属性的文档，则以后可以更好地做出有根据的猜测.

我可以计算出眼睛的宽度和两只眼睛之间的距离吗?

更新:在iOS 12中的ARKit"ARKit 2"中， rightEyeTransform 提供每个眼球中心的3D位置(相对于脸锚). (也可以是每只眼睛的方向.)这可能会对您的用例有所帮助，但是如果您实际上要做的事情与瞳孔的位置/大小或眼孔的大小有关……

没有API可以帮您做到这一点. ARKit确实提供了您可以用来自行找到信息的信息，但并不能保证始终有效.

ARFaceGeometry 为您提供了一个三角形网格，可映射数百个左右在会话中以拓扑稳定的方式指向面部.也就是说，例如，假设网格中的第57个顶点是鼻子的尖端，那么即使脸部起皱和伸展并且该点相对于其他点的位置发生变化，它也会停留在鼻子的尖端.

问题:

• API不会告诉您哪个顶点(网格中的点)是哪个顶点(就诸如眼角，鼻尖等脸部界标而言).
• 网格的拓扑在会话中是稳定的，但是Apple不保证在iOS版本，设备等之间不会更改.

因此，尽管进行了一些实验，您也许可以算出哪些顶点是左眼的内角，左眼的外角等.完成这些操作后，您可以查看它们的位置以估算出有用的像眼睛的宽度，瞳孔之间的距离等数量.但是，这些测量是基于关于网格的假设，这些假设可能并不总是成立，因此您不知道它何时会中断您的应用程序用户.

I am trying to get width of the eyes and distance of 2 eyes using 3D Face Mesh of ARKit.

I have used CATransform3D of ARAnchor;

 struct CATransform3D
{
  CGFloat m11, m12, m13, m14;
  CGFloat m21, m22, m23, m24;
  CGFloat m31, m32, m33, m34;
  CGFloat m41, m42, m43, m44;
};

Below is my code;

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {

guard let faceAnchor = anchor as? ARFaceAnchor else { return }

let leftcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkLeft]?.caTransform3DValue)!

let rightcaTransform3DValue : CATransform3D = (faceAnchor.blendShapes[.eyeBlinkRight]?.caTransform3DValue)!

print("  m11 : \(String(describing:leftcaTransform3DValue.m11)) m12 : \(String(describing:leftcaTransform3DValue.m12)) m13 : \(String(describing:leftcaTransform3DValue.m13)) m14 : \(String(describing:leftcaTransform3DValue.m14)) m21 : \(String(describing:leftcaTransform3DValue.m21)) m22 : \(String(describing:leftcaTransform3DValue.m22)) m23 : \(String(describing:leftcaTransform3DValue.m23)) m24 : \(String(describing:leftcaTransform3DValue.m24)) m31 : \(String(describing:leftcaTransform3DValue.m31)) m32 : \(String(describing:leftcaTransform3DValue.m32)) m33 : \(String(describing:leftcaTransform3DValue.m33)) m34 : \(String(describing:leftcaTransform3DValue.m34)) m41 : \(String(describing:leftcaTransform3DValue.m41)) m42 : \(String(describing:leftcaTransform3DValue.m42)) m43 : \(String(describing:leftcaTransform3DValue.m43)) m44 : \(String(describing:leftcaTransform3DValue.m44)) " )
}

And as a result of leftcaTransform3DValue I got values like;

m11 =  -5.22553711590422e-315
...
...
...
m44 =   2.13285635582599e-314

Same for the rightcaTransform3DValue.

So my question is do these values specify any dimension or size measurement?

Can I calculate width of the eyes and distance between two eyes?

Any help is really appreciated.

解决方案

So my question is do these values specify any dimension or size measurement?

No. The numbers you’re getting are nonsense, because the way you’re getting them is... maybe not quite nonsense, but pretty close.

The blendShapes dictionary on ARFaceAnchor is documented as having values of type NSNumber, where the underlying numeric value of the NSNumber is a Float between 0.0 and 1.0.

NSNumber is an object wrapper for many possible types of scalar numeric values. It has methods for fetching its underlying value as various types (by converting to a different representation of the same number). But given that these particular numbers are documented to be floating-point values between 0 and 1, there’s not much sense to fetching the intValue or boolValue, etc.

NSNumber is a subclass of NSValue, which is an object wrapper for many kinds of types not otherwise expressible as objects — ranges, sizes, pointers, and 3D transform matrices, among others. Those types can’t be converted between each other like numbers can, so the only type that’s meaningful to fetch out of an NSValue is the type it was created with. Any other type gives you nonsense.

Back to blendShapes — it’s further documented that each blend shape value in the dictionary isn’t just a number, but a number that tells you the progress of an animation parameter. eyeBlinkLeft doesn’t claim to tell you anything about where or how big the left eye is — it tells you how "blinked" (closed) the left eyelid is.

You’re barking up the wrong tree, but if you look at the docs for the classes and properties you’re using, you’ll be better able to make educated guesses later.

Can I calculate width of the eyes and distance between two eyes?

Update: In "ARKit 2", aka ARKit in iOS 12, the leftEyeTransform and rightEyeTransform provide the 3D position (relative to the face anchor) of the center of each eyeball. (Also the orientation of each eye.) That might help your use case, but if what you're actually after has to do with the position/size of the pupils or the eye openings...

There’s no API that’ll do this for you. ARKit does provide information you could use to find it out yourself, but not in a way that’s guaranteed to always work.

ARFaceGeometry gives you a triangle mesh that maps a few hundred or so points on the face, in a way that’s topologically stable within a session. That is, for example, assuming the 57th vertex in the mesh is the tip of the nose, it’ll stay on the tip of the nose even as the face wrinkles and stretches and that point’s position relative to other points changes.

The problems:

• The API doesn’t tell you which vertices (points in the mesh) are which (in terms of face landmarks like eye corner, nose tip, etc).
• The topology of the mesh is stable within a session, but Apple doesn’t guarantee it won’t change between iOS versions, devices, etc.

So, though some experimentation you might be able to work out which vertices are the inside corner of the left eye, the outside corner of the left eye, etc. Once you do that, you can look at their positions to estimate useful quantities like eye width, inter-pupillary distance, etc. However, those measurements are based on assumptions about the mesh that might not always hold, so you don’t know when it’ll break for users of your app.

这篇关于可以使用CATransform3D获取Face Mesh中的眼睛大小尺寸吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！