如何量化优化的 tflite 模型的输入和输出 [英] How to quantize inputs and outputs of optimized tflite model

问题描述

我使用以下代码生成量化的tflite模型

I use the following code to generate a quantized tflite model

import tensorflow as tf

def representative_dataset_gen():
  for _ in range(num_calibration_steps):
    # Get sample input data as a numpy array in a method of your choosing.
    yield [input]

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
tflite_quant_model = converter.convert()

但根据训练后量化:

生成的模型将被完全量化，但为了方便仍采用浮动输入和输出.

The resulting model will be fully quantized but still take float input and output for convenience.

要为 Google Coral Edge TPU 编译 tflite 模型，我还需要量化输入和输出.

To compile tflite model for Google Coral Edge TPU I need quantized input and output as well.

在模型中，我看到第一个网络层将浮点输入转换为 input_uint8，最后一层将 output_uint8 转换为浮点输出.如何编辑 tflite 模型以去除第一个和最后一个浮动图层?

In the model, I see that the first network layer converts float input to input_uint8 and the last layer converts output_uint8 to the float output. How do I edit tflite model to get rid of the first and last float layers?

我知道我可以在转换过程中将输入和输出类型设置为 uint8，但这与任何优化都不兼容.唯一可用的选择是使用伪量化，这会导致模型错误.

I know that I could set input and output type to uint8 during conversion, but this is not compatible with any optimizations. The only available option then is to use fake quantization which results in a bad model.

推荐答案

您可以通过设置 inference_input_type 和 inference_output_type (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/python/lite.py#L460-L476) 到 int8.

You can avoid the float to int8 and int8 to float "quant/dequant" op by setting inference_input_type and inference_output_type (https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/python/lite.py#L460-L476) to int8.

这篇关于如何量化优化的 tflite 模型的输入和输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！