Coral Edge TPU编译器无法转换tflite模型:模型未量化 [英] Coral Edge TPU Compiler cannot convert tflite model: Model not quantized

问题描述

我正在尝试使用TensorFlow lite部署一个简单的测试应用程序.我想在设备上使用Coral Edge TPU棒，因此我必须执行量化意识培训.我想适合一个功能f(x) = 2 x - 1.我的训练代码如下:

I am trying to deploy a simple test application with TensorFlow lite. I want to use the Coral Edge TPU Stick on my device, so I have to perform Quantization Aware Training. I want to fit a function f(x) = 2 x - 1. My training code looks like this:

import tensorflow as tf
import numpy as np
from tensorflow import keras
from tensorflow.contrib import lite

# Create model
model = keras.models.Sequential([keras.layers.Dense(units=1, input_shape=[1])])

# Quantization aware training
sess = keras.backend.get_session()
tf.contrib.quantize.create_training_graph(sess.graph)
sess.run(tf.global_variables_initializer())

tf.summary.FileWriter('logs/', graph=sess.graph)

model.compile(optimizer='sgd', loss='mean_squared_error')

# Training data
xs = np.array([ -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([ -3.0, -1.0, 1.0, 3.0, 5.0, 7.0], dtype=float)

model.fit(xs, ys, epochs=500, batch_size=2)

# Test the model for plausbility
print(model.predict([10.0]))

# Display the quantization-relevant variables
for node in sess.graph.as_graph_def().node:
    if 'weights_quant/AssignMaxLast' in node.name \
        or 'weights_quant/AssignMinLast' in node.name:
        tensor = sess.graph.get_tensor_by_name(node.name + ':0')
        print('{} = {}'.format(node.name, sess.run(tensor)))


# Save the keras model
keras_file = 'quant_linear.h5'
keras.models.save_model(model, keras_file)

# Convert the keras model into a tflite model
converter = lite.TocoConverter.from_keras_model_file(keras_file)
converter.post_training_quantize = True
tflite_model = converter.convert()
open('quant_linear.tflite', 'wb').write(tflite_model)

作为输出，我得到(省略了keras和CUDA的特定输出):

As output, I get (keras and CUDA specific output is omitted):

[[18.86733]]
dense/weights_quant/AssignMinLast = 0.0
dense/weights_quant/AssignMaxLast = 1.984399676322937

这里需要注意的两件事:该模型是合理的，它应该输出接近19的值.显然，它也使用量化权重.如果我未启用量化意识训练，则这两个变量将不会显示.

Two things to note here: the model is plausible, it should output a value close to 19. Obviously, it also uses quantized weights. If I do not enable quantization aware training, the two variables won't show up.

此外，此模型可以由tf-lite解释器实例加载和执行.为了能够在TPU支持下使用它，我必须使用tpuedge_compiler对其进行转换.安装后，我执行

Additionally, this model can be loaded and executed by a tf-lite interpreter instance. To be able to use it with TPU support, however, I have to convert it with the tpuedge_compiler. After installing it, I execute

edgetpu_compiler quant_linear.tflite

不幸的是，似乎无法识别该模型已量化.输出

Unfortunately, it seems to be unable to recognize that the model is quantized. It outputs

user@ubuntu:~/TensorFlow$ edgetpu_compiler quant_linear.tflite 
Edge TPU Compiler version 1.0.249710469
INFO: Initialized TensorFlow Lite runtime.
Invalid model: quant_linear.tflite
Model not quantized

我试图在线编译它，但也失败了.这是错误还是在培训/转换过程中我弄乱了东西?另外，也许有工具可以验证我是否确实使用了量化模型?

I have tried to compile it online, which also fails. Is this a bug or did I mess something up during training/converting? Also, maybe there is tool to verify that I really use a quantized model?

谢谢！

推荐答案

您必须使用显式在TFLite转换AFAIK中进行量化. 量化Keras模型的代码示例:

You have to use explicit quantization during the TFLite conversion AFAIK. Code example which quantizes a Keras model:

dataset = tf.data.Dataset(...)

def generator():
    for item in dataset:
        image = # get image from dataset item
        yield [np.array([image.astype(np.float32)])]

converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = tf.lite.RepresentativeDataset(generator)

model = converter.convert()

这篇关于Coral Edge TPU编译器无法转换tflite模型:模型未量化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！