将最佳检查点还原到估算器tensorflow 2.x [英] Restore best checkpoint to an estimator tensorflow 2.x

问题描述

简而言之，我使用tensorflow Dataset API放置了一个数据输入管线.然后，我实现了一个用于使用keras进行分类的CNN模型，并将其转换为估计量.我用input_fn为估算者的Train和Eval Specs提供了输入，以提供用于训练和评估的输入数据.最后，我使用 tf.estimator.train_and_evaluate

Briefly, I put in place a data input pipline using tensorflow Dataset API. Then, I implemented a CNN model for classification using keras, which i converted to an estimator. I feeded my estimator Train and Eval Specs with my input_fn providing input data for training and evaluation. And as final step I launched the model training with tf.estimator.train_and_evaluate

def my_input_fn(tfrecords_path):

    dataset = (...)
    return batch_fbanks, batch_labels

def build_model():
    model = tf.keras.models.Sequential()
    model.add(...)
    model.compile(...)

    return model

model = build_model()

run_config=tf.estimator.RunConfig(model_dir,save_summary_steps=100,save_checkpoints_steps=1000)
estimator = tf.keras.estimator.model_to_estimator(model,config=run_config)

def serving_input_receiver_fn():
    inputs = {'Conv1_input': tf.compat.v1.placeholder(shape=[None, 11,120,1], dtype=tf.float32)}
    return tf.estimator.export.ServingInputReceiver(inputs, inputs)

exporter = tf.estimator.BestExporter(serving_input_receiver_fn, name="best_exporter", exports_to_keep=5)

train_spec_dnn = tf.estimator.TrainSpec(input_fn = lambda: my_input_fn(train_data_path),hooks=[hook])
eval_spec_dnn = tf.estimator.EvalSpec(input_fn = lambda: my_eval_input_fn(eval_data_path),exporters=exporter,start_delay_secs=0,throttle_secs=15)

tf.estimator.train_and_evaluate(estimator, train_spec_dnn, eval_spec_dnn)

我使用 tf.estimator.BestExporter 保存5个最佳检查点，如上所示.完成训练后，我想重新加载最佳模型并将其转换为估算器，以重新评估模型并预测新数据集.但是我的问题是将检查点还原到估计量.我尝试了几种解决方案，但是每次我没有获得estimator对象时，都需要运行其 evaluate 和 predict 方法.

I save the 5 best checkpoints using the tf.estimator.BestExporter as shown above. Once i finished training, i want to reload the best model and convert it to an estimator to re-evaluate the model and predict on new dataset. However my issue is in restoring the checkpoint to an estimator. I tried several solutions but each time i don't get the estimator object I need to run its evaluate and predict methods.

只需指定更多，每个最佳检查点目录的组织如下:

Just to specify more, each of the best checkpoints directory is organised as follow:

./
  variables/
      variables.data-00000-of-00002
      variables.data-00001-of-00002
      variables.index
  saved_model.pb

所以问题是如何从最佳检查点获取一个估计器对象，以便可以使用它来评估模型并预测新数据?

So the question is how can I get an estimator object from the best checkpoint so that i can use it to evaluate my model and predict on new data?

注意::我发现了一些依赖于TensorFlow v1功能的建议解决方案，这些功能无法解决我的问题，因为我使用的是TF v2.

Note : I found some proposed solutions relying on TensorFlow v1 features which can not solve my problem because i work with TF v2.

非常感谢，感谢您的帮助.

Thanks a lot, any help is appreciated.

推荐答案

您可以使用下面的从 tf.estimator.BestExporter

You can use the class below created from tf.estimator.BestExporter

它的作用是，除了保存最佳模型(.pb文件等)之外，还将保存 最佳导出模型检查点 在另一个文件夹上.

What it does is, except for saving the best model (.pb files and etc) it will also save the best-exported model checkpoint on a different folder.

下面是课程:

import shutil, glob, os
# import tensorflow.logging as logging
## the path where all the checkpoint reside
BEST_CHECKPOINTS_PATH_FROM = 'PATH TO ALL CHECKPOINT FILES'
## the path it will save the best exporter checkpoint files
BEST_CHECKPOINTS_PATH_TO = 'PATH TO BEST EXPORTER CHECKPOINT FILES TO BE SAVE' 

class BestCheckpointsExporter(tf.estimator.BestExporter):
      def export(self, estimator, export_path, checkpoint_path, eval_result,is_the_final_export):
          if self._best_eval_result is None or \
                    self._compare_fn(self._best_eval_result, eval_result):
                    #print('Exporting a better model ({} instead of {})...'.format(eval_result, self._best_eval_result))
              for name in glob.glob(checkpoint_path + '.*'):
                    print(name)
                    print(os.path.join(BEST_CHECKPOINTS_PATH_TO, os.path.basename(name)))
                    shutil.copy(name, os.path.join(BEST_CHECKPOINTS_PATH_TO, os.path.basename(name)))
                # also save the text file used by the estimator api to find the best checkpoint
              with open(os.path.join(BEST_CHECKPOINTS_PATH_TO, "checkpoint"), 'w') as f:
                    f.write("model_checkpoint_path: \"{}\"".format(os.path.basename(checkpoint_path)))
              self._best_eval_result = eval_result
          else:
              print('Keeping the current best model ({} instead of {}).'.format(self._best_eval_result, eval_result))

类用法的示例
您只需通过调用类来替换导出程序，然后传递serve_input_receiver_fn.

Example Usage of the Class
You will just replace the exporter by calling the class and pass the serving_input_receiver_fn.

def serving_input_receiver_fn():
    inputs = {'my_dense_input': tf.compat.v1.placeholder(shape=[None, 4], dtype=tf.float32)}
    return tf.estimator.export.ServingInputReceiver(inputs, inputs)

exporter = BestCheckpointsExporter(serving_input_receiver_fn=serving_input_receiver_fn) 

train_spec_dnn = tf.estimator.TrainSpec(input_fn = input_fn, max_steps=5)

eval_spec_dnn = tf.estimator.EvalSpec(input_fn=input_fn,exporters=exporter,start_delay_secs=0,throttle_secs=15)
(x, y) =  tf.estimator.train_and_evaluate(keras_estimator, train_spec_dnn, eval_spec_dnn)

这时，它将最佳导出的模型检查点文件保存在您指定的文件夹中.

At this point, It will save the best-exported model checkpoint files in the folder you have specified.

要加载检查点文件，您需要执行以下步骤:
第1步:重建模型实例

For loading the checkpoint files you need to do the following steps:
Step 1: Rebuild your model instance

def build_model():
    model = tf.keras.models.Sequential()
    model.add(...)
    model.compile(...)

    return model

model = build_model()

步骤2:使用模型load_weights API
参考网址: https://www.tensorflow.org/tutorials/keras/save_and_load

ck_path = tf.train.latest_checkpoint('PATH TO BEST EXPORTER CHECKPOINT FILES')
model.load_weights(ck_path)

## From there you will be able to call the predict & evaluate the functionality of the trained model

##PREDICT
prediction = model.predict(x)

##EVALUATE
for features_batch, labels_batch in input_fn().take(1):
  model.evaluate(features_batch, labels_batch)

注意:所有这些都已在google colab上进行了仿真.

Note: All of these have been simulated on google colab.

这篇关于将最佳检查点还原到估算器tensorflow 2.x的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！