无法保存自定义子类模型 [英] Can't save custom subclassed model

问题描述

受 tf.keras.Model 子类化的启发，我创建了自定义模型.
我可以训练它并获得成功的结果，但我无法保存它.
我使用 python3.6 和 tensorflow v1.10(或 v1.9)

Inspired by tf.keras.Model subclassing I created custom model.
I can train it and get successfull results, but I can't save it.
I use python3.6 with tensorflow v1.10 (or v1.9)

这里的最小完整代码示例:

Minimal complete code example here:

import tensorflow as tf
from tensorflow.keras.datasets import mnist


class Classifier(tf.keras.Model):
    def __init__(self):
        super().__init__(name="custom_model")

        self.batch_norm1 = tf.layers.BatchNormalization()
        self.conv1 = tf.layers.Conv2D(32, (7, 7))
        self.pool1 = tf.layers.MaxPooling2D((2, 2), (2, 2))

        self.batch_norm2 = tf.layers.BatchNormalization()
        self.conv2 = tf.layers.Conv2D(64, (5, 5))
        self.pool2 = tf.layers.MaxPooling2D((2, 2), (2, 2))

    def call(self, inputs, training=None, mask=None):
        x = self.batch_norm1(inputs)
        x = self.conv1(x)
        x = tf.nn.relu(x)
        x = self.pool1(x)

        x = self.batch_norm2(x)
        x = self.conv2(x)
        x = tf.nn.relu(x)
        x = self.pool2(x)

        return x


if __name__ == '__main__':
    (x_train, y_train), (x_test, y_test) = mnist.load_data()

    x_train = x_train.reshape(*x_train.shape, 1)[:1000]
    y_train = y_train.reshape(*y_train.shape, 1)[:1000]

    x_test = x_test.reshape(*x_test.shape, 1)
    y_test = y_test.reshape(*y_test.shape, 1)

    y_train = tf.keras.utils.to_categorical(y_train)
    y_test = tf.keras.utils.to_categorical(y_test)

    model = Classifier()

    inputs = tf.keras.Input((28, 28, 1))

    x = model(inputs)
    x = tf.keras.layers.Flatten()(x)
    x = tf.keras.layers.Dense(10, activation="sigmoid")(x)

    model = tf.keras.Model(inputs=inputs, outputs=x)
    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
    model.fit(x_train, y_train, epochs=1, shuffle=True)

    model.save("./my_model")

错误信息:

1000/1000 [==============================] - 1s 1ms/step - loss: 4.6037 - acc: 0.7025
Traceback (most recent call last):
  File "/home/user/Data/test/python/mnist/mnist_run.py", line 62, in <module>
    model.save("./my_model")
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1278, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/saving.py", line 101, in save_model
    'config': model.get_config()
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1049, in get_config
    layer_config = layer.get_config()
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1028, in get_config
    raise NotImplementedError
NotImplementedError

Process finished with exit code 1

我查看了错误行，发现 get_config 方法检查 self._is_graph_network

I looked into the error line and found out that get_config method checks self._is_graph_network

有人解决这个问题吗?

谢谢！

更新 1:
在 keras 2.2.2(不是 tf.keras)上
找到评论(用于模型保存)
文件:keras/engine/network.py
功能:get_config

Update 1:
On the keras 2.2.2 (not tf.keras)
Found comment (for model saving)
file: keras/engine/network.py
Function: get_config

# 子类网络不可序列化
# (除非序列化是由
实现的# 子类网络的作者).

# Subclassed networks are not serializable
# (unless serialization is implemented by
# the author of the subclassed network).

所以，显然它行不通...
我想知道，他们为什么不在 文档 中指出这一点(例如:使用子类化而不保存能力！")

So, obviously it won't work...
I wonder, why don't they point it out in the documentation (Like: "Use subclassing without ability to save!")

更新 2:
在 keras 文档中找到:

在子类模型中，模型的拓扑被定义为 Python 代码
(而不是作为层的静态图).这意味着模型的
拓扑不能被检查或序列化.结果，以下
方法和属性不适用于子类模型:

In subclassed models, the model's topology is defined as Python code
(rather than as a static graph of layers). That means the model's
topology cannot be inspected or serialized. As a result, the following
methods and attributes are not available for subclassed models:

模型.输入和模型.输出.
model.to_yaml() 和 model.to_json()
model.get_config() 和 model.save().

model.inputs and model.outputs.
model.to_yaml() and model.to_json()
model.get_config() and model.save().

因此，无法通过使用子类化来保存模型.
可以只使用 Model.save_weights()

推荐答案

TensorFlow 2.2

感谢@cal 通知我新的 TensorFlow 支持保存自定义模型！

TensorFlow 2.2

Thanks for @cal for noticing me that the new TensorFlow has supported saving the custom models!

通过使用model.save 来保存整个模型，并通过使用load_model 来恢复之前存储的子类模型.以下代码片段描述了如何实现它们.

By using model.save to save the whole model and by using load_model to restore previously stored subclassed model. The following code snippets describe how to implement them.

class ThreeLayerMLP(keras.Model):

  def __init__(self, name=None):
    super(ThreeLayerMLP, self).__init__(name=name)
    self.dense_1 = layers.Dense(64, activation='relu', name='dense_1')
    self.dense_2 = layers.Dense(64, activation='relu', name='dense_2')
    self.pred_layer = layers.Dense(10, name='predictions')

  def call(self, inputs):
    x = self.dense_1(inputs)
    x = self.dense_2(x)
    return self.pred_layer(x)

def get_model():
  return ThreeLayerMLP(name='3_layer_mlp')

model = get_model()
# Save the model
model.save('path_to_my_model',save_format='tf')

# Recreate the exact same model purely from the file
new_model = keras.models.load_model('path_to_my_model')

参见:使用 Keras 保存和序列化模型 - 第二部分:子类模型的保存和加载

TL;博士:

1. 不要将 model.save() 用于自定义子类 keras 模型；
2. 改用 save_weights() 和 load_weights().

1. do not use model.save() for custom subclass keras model;
2. use save_weights() and load_weights() instead.

<小时>

在 Tensorflow 团队的帮助下，事实证明保存自定义子类 Keras 模型的最佳做法是保存其权重并在需要时重新加载.

---

With the help of the Tensorflow Team, it turns out the best practice of saving a Custom Sub-Class Keras Model is to save its weights and load it back when needed.

我们不能简单地保存一个 Keras 自定义子类模型的原因是它包含自定义代码，不能安全地序列化.但是，当我们具有相同的模型结构和自定义代码时，可以毫无问题地保存/加载权重.

The reason that we can not simply save a Keras custom subclass model is that it contains custom codes, which can not be serialized safely. However, the weights can be saved/loaded when we have the same model structure and custom codes without any problem.

Keras 的作者 Francois Chollet 写了一篇很棒的教程，介绍了如何在 Colab 中的 Tensorflow 2.0 中保存/加载 Sequential/Functional/Keras/Custom Sub-Class Models at 此处.在保存子类模型部分，它说:

There has a great tutorial written by Francois Chollet who is the author of Keras, for how to save/load Sequential/Functional/Keras/Custom Sub-Class Models in Tensorflow 2.0 in Colab at here. In Saving Subclassed Models section, it said that:

顺序模型和功能模型是表示层的 DAG 的数据结构.因此，它们可以安全地序列化和反序列化.

Sequential models and Functional models are datastructures that represent a DAG of layers. As such, they can be safely serialized and deserialized.

子类模型的不同之处在于它不是数据结构，而是一个一段代码.模型的架构是通过主体定义的调用方法.这意味着模型的架构不能安全地序列化.要加载模型，您需要具有访问创建它的代码(模型子类的代码).或者，您可以将此代码序列化为字节码(例如通过酸洗)，但这是不安全的，通常不可移植.

A subclassed model differs in that it's not a datastructure, it's a piece of code. The architecture of the model is defined via the body of the call method. This means that the architecture of the model cannot be safely serialized. To load a model, you'll need to have access to the code that created it (the code of the model subclass). Alternatively, you could be serializing this code as bytecode (e.g. via pickling), but that's unsafe and generally not portable.

这篇关于无法保存自定义子类模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！