MediaPipe TensorflowLite Iris模型 [英] MediaPipe TensorflowLite Iris Model

问题描述

我正在尝试了解tflite Iris的输出地标模型可从mediapipe获得.

I am trying to understand the output of the tflite Iris landmarks model available from mediapipe.

模型卡将输出描述为71个2D界标和5个2D界标.在检查模型时，如下所示:

The model card describes the output as 71 2D landmarks and 5 2D landmarks. When inspecting the model as follows:

interpreter = tf.lite.Interpreter(model_path='iris_landmark.tflite')
interpreter.allocate_tensors()
output_details = interpreter.get_output_details()

print(output_details)
[{'dtype': numpy.float32,
  'index': 384,
  'name': 'output_eyes_contours_and_brows',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
   'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32)},
  'shape': array([  1, 213], dtype=int32),
  'shape_signature': array([  1, 213], dtype=int32),
  'sparsity_parameters': {}},
 {'dtype': numpy.float32,
  'index': 385,
  'name': 'output_iris',
  'quantization': (0.0, 0),
  'quantization_parameters': {'quantized_dimension': 0,
   'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32)},
  'shape': array([ 1, 15], dtype=int32),
  'shape_signature': array([ 1, 15], dtype=int32),
  'sparsity_parameters': {}}]

我在模型输出中看到213个值和15个值-因此，我假设我每个点都获得了x/y/z坐标.在图像上运行模型后，我得到的值在-7000到+7000范围内.我输入的是64x64的图片，您知道这些点如何与原始图片相对应吗?

I see 213 values and 15 values in the model outputs - so I assume I am getting an x/y/z coordinate for each point. After running the model on an image I get values in the -7000 to +7000 range. My input was a 64x64 image, any idea of how these points correspond to the original image?

我想获得在mediapipe示例中呈现的眼睛关键点的像素坐标.

I would like to have pixel coordinates of the eye keypoints, which are rendered in the mediapipe examples.

推荐答案

模型卡似乎是错误的，它实际上输出3D坐标，在模型输入和输出上还存在一些尚不清楚的规范化，但是用于绘制2d地标.

The model card appears to be wrong, it actually outputs 3D coordinates, there are also some normalization on the model input and output that isn't clear, but is used for drawing the 2d landmarks.

我在我的发现中在这里打开了一个github问题.我还没有看到与模型卡相关的任何更改.

I opened a github issue with my findings here. I haven't seen any changes related to the model card.

在此处，我创建了一个演示正确用法的协作实验室.您可以忽略z坐标，而将x/y坐标绘制到图像上以查看界标.

I created a colab demonstrating proper usage, here. You can ignore the z coordinate and plot the x/y coordinates onto your image to see the landmarks.

我可能应该用虹膜图片示例更新colab.

I probably should update the colab with an iris picture example.

这篇关于MediaPipe TensorflowLite Iris模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！