使用 toco 将伪量化 tensorflow 模型(.pb)转换为 tensorflow lite 模型(.tflite)失败 [英] convert fake quantized tensorflow model(.pb) to tensorflow lite model(.tflite) using toco failed

问题描述

我尝试按照张量流量化中的说明生成量化的张量流精简版.

I tried to follow instructions in tensorflow quantization to generate a quantized tensorflow lite model.

首先，我在训练过程中使用 tf.contrib.quantize.create_training_graph() 和 tf.contrib.quantize.create_eval_graph() 将伪量化节点插入图中，并生成一个冻结的 pb 文件(model.pb)终于.

First, I use tf.contrib.quantize.create_training_graph() and tf.contrib.quantize.create_eval_graph() in my training process to insert fake quantization node into the graph, and generate a frozen pb file(model.pb) finally.

其次，我使用以下命令将我的伪量化 tensorflow 模型转换为量化的 tensorflow lite 模型.

Second, I use following command to convert my fake quantized tensorflow model to quantized tensorflow lite model.

bazel-bin/tensorflow/contrib/lite/toco/toco \
--input_file=model.pb \
--input_format=TENSORFLOW_GRAPHDEF \
--output_format=TFLITE \
--output_file=model.tflite \
--inference_type=QUANTIZED_UINT8 --input_shapes=1,1:1,5002 \
--input_arrays=Test/Model/input,Test/Model/apps \
--output_arrays=Test/Model/output_probs,Test/Model/final_state  \
--mean_values=127.5,127.5 --std_values=127.5,127.5 --allow_custom_ops

隐蔽过程失败，日志如下:

The covert process failed with following logs:

2018-03-28 18:00:38.348403: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before Removing unused ops: 118 operators, 193 arrays (0 quantized)
2018-03-28 18:00:38.349394: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before general graph transformations: 118 operators, 193 arrays (0 quantized)
2018-03-28 18:00:38.382854: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 1: 57 operators, 103 arrays (1 quantized)
2018-03-28 18:00:38.384327: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 2: 56 operators, 101 arrays (1 quantized)
2018-03-28 18:00:38.385235: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After general graph transformations pass 3: 55 operators, 100 arrays (1 quantized)
2018-03-28 18:00:38.385995: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] Before pre-quantization graph transformations: 55 operators, 100 arrays (1 quantized)
2018-03-28 18:00:38.386047: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:131] Skipping min-max setting for {TensorFlowSplit operator with output Test/Model/RNN/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/split} because output Test/Model/RNN/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/split already has min-max.
2018-03-28 18:00:38.386076: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:131] Skipping min-max setting for {TensorFlowSplit operator with output Test/Model/RNN/RNN/multi_rnn_cell/cell_1/basic_lstm_cell/split} because output Test/Model/RNN/RNN/multi_rnn_cell/cell_1/basic_lstm_cell/split already has min-max.
2018-03-28 18:00:38.386328: I tensorflow/contrib/lite/toco/graph_transformations/graph_transformations.cc:39] After pre-quantization graph transformations pass 1: 48 operators, 93 arrays (1 quantized)
2018-03-28 18:00:38.386484: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:131] Skipping min-max setting for {TensorFlowSplit operator with output Test/Model/RNN/RNN/multi_rnn_cell/cell_1/basic_lstm_cell/split} because output Test/Model/RNN/RNN/multi_rnn_cell/cell_1/basic_lstm_cell/split already has min-max.
2018-03-28 18:00:38.386502: W tensorflow/contrib/lite/toco/graph_transformations/hardcode_min_max.cc:131] Skipping min-max setting for {TensorFlowSplit operator with output Test/Model/RNN/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/split} because output Test/Model/RNN/RNN/multi_rnn_cell/cell_0/basic_lstm_cell/split already has min-max.
2018-03-28 18:00:38.386778: F tensorflow/contrib/lite/toco/tooling_util.cc:1432] Array Test/Model/embedding_lookup, which is an input to the TensorFlowReshape operator producing the output array Test/Model/Reshape_1, is lacking min/max data, which is necessary for quantization. Either target a non-quantized output format, or change the input graph to contain min/max information, or pass --default_ranges_min= and --default_ranges_max= if you do not care about the accuracy of results.
Aborted

有什么问题，我哪里错了?

What is the problem and where am I wrong?

推荐答案

你没有做错任何事.

目前 create_training_graph 和 create_eval_graph 在各种模型架构中并不是最健壮的.我们让他们在大多数 CNN 上工作，但 RNN 仍在进行中并提出了一系列不同的挑战.

Currently create_training_graph and create_eval_graph aren't the most robust over various model architectures. We have them working on most CNNs but RNNs are still in progress and pose a different set of challenges.

根据 RNN 的细节，现在的量化方法将更加复杂，可能需要手动将 FakeQuantization 操作放在正确的位置.特别是在您的错误消息中，您似乎需要在 embedding_lookup 中添加 FakeQuantization 操作.话虽如此，最终的量化 RNN 可能会运行，但我不知道准确度会如何.它最终取决于模型和数据集:)

Depending on the details of the RNN, the approach for quantization right now will be more involved and may require manually putting FakeQuantization operations in the correct places. Specifically in your error message it seems that you need to add a FakeQuantization operation at the embedding_lookup. That being said, the final quantized RNN may run, but I have no idea how the accuracy will look. It really ends up being model and dataset dependent :)

当自动重写正确支持 RNN 时，我将更新此答案.

I will update this answer when RNNs are properly supported by the automatic rewrites.

这篇关于使用 toco 将伪量化 tensorflow 模型(.pb)转换为 tensorflow lite 模型(.tflite)失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！