量化Keras神经网络模型 [英] Quantize a Keras neural network model

问题描述

最近，我开始使用Tensorflow + Keras创建神经网络，我想尝试一下Tensorflow中可用的量化功能.到目前为止，尝试使用TF教程中的示例都很好，并且我有这个基本的示例(来自 https: //www.tensorflow.org/tutorials/keras/basic_classification ):

Recently, I've started creating neural networks with Tensorflow + Keras and I would like to try the quantization feature available in Tensorflow. So far, experimenting with examples from TF tutorials worked just fine and I have this basic working example (from https://www.tensorflow.org/tutorials/keras/basic_classification):

import tensorflow as tf
from tensorflow import keras

fashion_mnist = keras.datasets.fashion_mnist
(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

# fashion mnist data labels (indexes related to their respective labelling in the data set)
class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat', 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

# preprocess the train and test images
train_images = train_images / 255.0
test_images = test_images / 255.0

# settings variables
input_shape = (train_images.shape[1], train_images.shape[2])

# create the model layers
model = keras.Sequential([
keras.layers.Flatten(input_shape=input_shape),
keras.layers.Dense(128, activation=tf.nn.relu),
keras.layers.Dense(10, activation=tf.nn.softmax)
])

# compile the model with added settings
model.compile(optimizer=tf.train.AdamOptimizer(),
          loss='sparse_categorical_crossentropy',
          metrics=['accuracy'])

# train the model
epochs = 3
model.fit(train_images, train_labels, epochs=epochs)

# evaluate the accuracy of model on test data
test_loss, test_acc = model.evaluate(test_images, test_labels)
print('Test accuracy:', test_acc)

现在，我想在学习和分类过程中采用量化.量化文档( https://www.tensorflow.org/performance/quantization )(页面自cca 2018年9月15日起不再可用)建议使用以下代码:

Now, I would like to employ quantization in the learning and classification process. The quantization documentation (https://www.tensorflow.org/performance/quantization) (the page is no longer available since cca September 15, 2018) suggests to use this piece of code:

loss = tf.losses.get_total_loss()
tf.contrib.quantize.create_training_graph(quant_delay=2000000)
optimizer = tf.train.GradientDescentOptimizer(0.00001)
optimizer.minimize(loss)

但是，它不包含有关在何处使用此代码或如何将其连接到TF代码的任何信息(甚至没有提及使用Keras创建的高级模型).我不知道这个量化部分与先前创建的神经网络模型之间的关系.只需将其插入神经网络代码，就会遇到以下错误:

However, it does not contain any information about where this code should be utilized or how it should be connected to a TF code (not even mentioning a high level model created with Keras). I have no idea how this quantization part relates to the previously created neural network model. Just inserting it following the neural network code runs into the following error:

Traceback (most recent call last):
  File "so.py", line 41, in <module>
    loss = tf.losses.get_total_loss()
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/losses/util.py", line 112, in get_total_loss
    return math_ops.add_n(losses, name=name)
  File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/math_ops.py", line 2119, in add_n
    raise ValueError("inputs must be a list of at least one Tensor with the "
ValueError: inputs must be a list of at least one Tensor with the same dtype and shape

是否可以通过这种方式量化Keras NN模型，或者我是否缺少一些基本知识? 可能想到的解决方案是使用低级TF API代替Keras(需要做大量工作来构建模型)，或者尝试从Keras模型中提取一些较低级的方法.

Is it possible to quantize a Keras NN model in this way or am I missing something basic? A possible solution that crossed my mind could be using low level TF API instead of Keras (needing to do quite a bit of work to construct the model), or maybe trying to extract some of the lower level methods from the Keras models.

推荐答案

如其他答案所述，TensorFlow Lite可以帮助您进行网络量化.

As mentioned in other answers, TensorFlow Lite can help you with network quantization.

TensorFlow Lite提供了多个级别的量化支持.

TensorFlow Lite provides several levels of support for quantization.

Tensorflow Lite训练后量化量化权重， 激活后易于训练.量化意识训练可以 用于训练可以以最小的精度量化的网络 降低;这仅适用于卷积神经的子集 网络体系结构.

Tensorflow Lite post-training quantization quantizes weights and activations post training easily. Quantization-aware training allows for training of networks that can be quantized with minimal accuracy drop; this is only available for a subset of convolutional neural network architectures.

因此，首先，您需要确定是否需要训练后量化还是了解量化的培训.例如，如果您已经将模型另存为* .h5文件，则可能要遵循@Mitiku的说明并进行训练后量化.

So first, you need to decide whether you need post-training quantization or quantization-aware training. For example, if you already saved the model as *.h5 files, you would probably want to follow @Mitiku's instruction and do the post-training quantization.

如果您希望通过模拟训练中的量化效果(使用问题中引用的方法)来获得更高的性能，并且您的模型位于受量化支持的CNN体​​系结构子集中，有意识的培训，此示例可以在Keras和TensorFlow之间的交互方面为您提供帮助.基本上，您只需要在模型定义及其拟合之间添加以下代码:

If you prefer to achieve higher performance by simulating the effect of quantization in training (using the method you quoted in the question), and your model is in the subset of CNN architecture supported by quantization-aware training, this example may help you in terms of interaction between Keras and TensorFlow. Basically, you only need to add this code between model definition and its fitting:

sess = tf.keras.backend.get_session()
tf.contrib.quantize.create_training_graph(sess.graph)
sess.run(tf.global_variables_initializer())

这篇关于量化Keras神经网络模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！