无法保存自定义子类化模型 [英] Can't save custom subclassed model

问题描述

受 tf.keras.Model子类化的启发，我创建了自定义模型.
我可以训练它并获得成功的结果，但是我无法保存.
我在tensorflow v1.10(或v1.9)中使用python3.6

Inspired by tf.keras.Model subclassing I created custom model.
I can train it and get successfull results, but I can't save it.
I use python3.6 with tensorflow v1.10 (or v1.9)

此处是最小的完整代码示例:

Minimal complete code example here:

import tensorflow as tf
from tensorflow.keras.datasets import mnist


class Classifier(tf.keras.Model):
    def __init__(self):
        super().__init__(name="custom_model")

        self.batch_norm1 = tf.layers.BatchNormalization()
        self.conv1 = tf.layers.Conv2D(32, (7, 7))
        self.pool1 = tf.layers.MaxPooling2D((2, 2), (2, 2))

        self.batch_norm2 = tf.layers.BatchNormalization()
        self.conv2 = tf.layers.Conv2D(64, (5, 5))
        self.pool2 = tf.layers.MaxPooling2D((2, 2), (2, 2))

    def call(self, inputs, training=None, mask=None):
        x = self.batch_norm1(inputs)
        x = self.conv1(x)
        x = tf.nn.relu(x)
        x = self.pool1(x)

        x = self.batch_norm2(x)
        x = self.conv2(x)
        x = tf.nn.relu(x)
        x = self.pool2(x)

        return x


if __name__ == '__main__':
    (x_train, y_train), (x_test, y_test) = mnist.load_data()

    x_train = x_train.reshape(*x_train.shape, 1)[:1000]
    y_train = y_train.reshape(*y_train.shape, 1)[:1000]

    x_test = x_test.reshape(*x_test.shape, 1)
    y_test = y_test.reshape(*y_test.shape, 1)

    y_train = tf.keras.utils.to_categorical(y_train)
    y_test = tf.keras.utils.to_categorical(y_test)

    model = Classifier()

    inputs = tf.keras.Input((28, 28, 1))

    x = model(inputs)
    x = tf.keras.layers.Flatten()(x)
    x = tf.keras.layers.Dense(10, activation="sigmoid")(x)

    model = tf.keras.Model(inputs=inputs, outputs=x)
    model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
    model.fit(x_train, y_train, epochs=1, shuffle=True)

    model.save("./my_model")

错误消息:

1000/1000 [==============================] - 1s 1ms/step - loss: 4.6037 - acc: 0.7025
Traceback (most recent call last):
  File "/home/user/Data/test/python/mnist/mnist_run.py", line 62, in <module>
    model.save("./my_model")
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1278, in save
    save_model(self, filepath, overwrite, include_optimizer)
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/saving.py", line 101, in save_model
    'config': model.get_config()
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1049, in get_config
    layer_config = layer.get_config()
  File "/home/user/miniconda3/envs/ml3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/network.py", line 1028, in get_config
    raise NotImplementedError
NotImplementedError

Process finished with exit code 1

我查看了错误行，发现 get_config 方法检查 self._is_graph_network

I looked into the error line and found out that get_config method checks self._is_graph_network

有人处理这个问题吗?

谢谢！

更新1:
在keras 2.2.2上(不是tf.keras)
找到评论(用于模型保存)
文件:keras/engine/network.py
函数:get_config

Update 1:
On the keras 2.2.2 (not tf.keras)
Found comment (for model saving)
file: keras/engine/network.py
Function: get_config

#子类网络不可序列化
#(除非由
#子类别网络的作者).

# Subclassed networks are not serializable
# (unless serialization is implemented by
# the author of the subclassed network).

所以，显然这是行不通的...
我想知道，他们为什么不在文档中指出(例如:保存的能力！")

So, obviously it won't work...
I wonder, why don't they point it out in the documentation (Like: "Use subclassing without ability to save!")

更新2:
在 keras文档中找到:

Update 2:
Found in keras documentation:

在子类化模型中，模型的拓扑定义为Python代码
(而不是作为图层的静态图).这意味着该模型的
无法检查或序列化拓扑.结果，以下
方法和属性不适用于子模型:

In subclassed models, the model's topology is defined as Python code
(rather than as a static graph of layers). That means the model's
topology cannot be inspected or serialized. As a result, the following
methods and attributes are not available for subclassed models:

model.inputs和model.outputs.
model.to_yaml()和model.to_json()
model.get_config()和model.save().

model.inputs and model.outputs.
model.to_yaml() and model.to_json()
model.get_config() and model.save().

因此，无法使用子类保存模型.
只能使用Model.save_weights()

So, there is no way to save model by using subclassing.
It's possible to only use Model.save_weights()

推荐答案

TensorFlow 2.2

感谢@cal通知我新的TensorFlow支持保存自定义模型！

TensorFlow 2.2

Thanks for @cal for noticing me that the new TensorFlow has supported saving the custom models!

通过使用model.save保存整个模型，并通过使用load_model恢复以前存储的子类模型.以下代码段描述了如何实现它们.

By using model.save to save the whole model and by using load_model to restore previously stored subclassed model. The following code snippets describe how to implement them.

class ThreeLayerMLP(keras.Model):

  def __init__(self, name=None):
    super(ThreeLayerMLP, self).__init__(name=name)
    self.dense_1 = layers.Dense(64, activation='relu', name='dense_1')
    self.dense_2 = layers.Dense(64, activation='relu', name='dense_2')
    self.pred_layer = layers.Dense(10, name='predictions')

  def call(self, inputs):
    x = self.dense_1(inputs)
    x = self.dense_2(x)
    return self.pred_layer(x)

def get_model():
  return ThreeLayerMLP(name='3_layer_mlp')

model = get_model()
# Save the model
model.save('path_to_my_model',save_format='tf')

# Recreate the exact same model purely from the file
new_model = keras.models.load_model('path_to_my_model')

请参阅:使用Keras保存和序列化模型-第二部分:子类的保存和加载型号

TL; DR:

1. 请勿将model.save()用于自定义子类keras模型；
2. 改为使用save_weights()和load_weights().

1. do not use model.save() for custom subclass keras model;
2. use save_weights() and load_weights() instead.

---

在Tensorflow团队的帮助下，保存自定义子类Keras模型的最佳实践是保存其权重并在需要时将其加载回去.

---

With the help of the Tensorflow Team, it turns out the best practice of saving a Custom Sub-Class Keras Model is to save its weights and load it back when needed.

我们之所以不能简单地保存Keras自定义子类模型，是因为它包含自定义代码，无法安全序列化.但是，当我们具有相同的模型结构和自定义代码时，可以保存/加载权重.

The reason that we can not simply save a Keras custom subclass model is that it contains custom codes, which can not be serialized safely. However, the weights can be saved/loaded when we have the same model structure and custom codes without any problem.

由Keras的作者Francois Chollet撰写的精彩教程介绍了如何在Colab的Tensorflow 2.0中保存/加载Sequential/Functional/Keras/Custom子类模型，网址为

There has a great tutorial written by Francois Chollet who is the author of Keras, for how to save/load Sequential/Functional/Keras/Custom Sub-Class Models in Tensorflow 2.0 in Colab at here. In Saving Subclassed Models section, it said that:

顺序模型和功能模型是表示层DAG的数据结构.这样，它们可以安全地序列化和反序列化.

Sequential models and Functional models are datastructures that represent a DAG of layers. As such, they can be safely serialized and deserialized.

子类化模型的不同之处在于，它不是数据结构，而是数据结构. 一段代码.通过主体定义模型的架构 调用方法.这意味着模型的架构 无法安全地序列化.要加载模型，您需要 访问创建它的代码(模型子类的代码). 或者，您可以将此代码序列化为字节码(例如 通过酸洗)，但这是不安全的，而且通常不方便携带.

A subclassed model differs in that it's not a datastructure, it's a piece of code. The architecture of the model is defined via the body of the call method. This means that the architecture of the model cannot be safely serialized. To load a model, you'll need to have access to the code that created it (the code of the model subclass). Alternatively, you could be serializing this code as bytecode (e.g. via pickling), but that's unsafe and generally not portable.

这篇关于无法保存自定义子类化模型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！