从 Tensorflow Keras 检查点重新加载最佳权重 [英] Reload best weights from Tensorflow Keras Checkpoints
问题描述
有没有办法在训练结束后重新加载某个时期的权重或 ModelCheckpoint
创建的模型检查点文件中的最佳权重?
Is there a way to reload the weights from a certain epoch or the best weights from the model checkpoint files created by ModelCheckpoint
once the training is over?
我已经训练了 10 个 epoch 并创建了一个检查点,该检查点仅在每个 epoch 之后保存权重.最后一个 epoch 的 val_categorical_accuracy 比 epoch no 低一点.5. 我知道我应该设置 save_best_only=True
但我错过了.
I have trained that trained for 10 epochs and created a checkpoint that only saved weights after each epoch. The final epoch's val_categorical_accuracy is a bit lower than epoch no. 5. I know I should have set save_best_only=True
but I missed that.
- 那么,现在有没有办法从最好的时代或第 5 个时代获得权重?
- 此外,
ModelCheckpoint
是否在每个 epoch 之后覆盖权重检查点文件?
- So now, is there a way to get the weights from the best epoch or the epoch number 5?
- Also, does
ModelCheckpoint
overwrites weights after each epoch in the checkpoint file?
我有哪些选择?提前感谢您的帮助.
What are my options here? Thanks for your help in advance.
以下是我的实现:
checkpoint_path = 'saved_model/cp.ckpt'
checkpoint_dir = os.path.dirname(checkpoint_path)
print(checkpoint_dir)
lstm_model.fit(X_train_seq_pad, y_train_cat,
epochs=100,
validation_data=(X_val_seq_pad, y_val_cat),
callbacks=[callbacks.EarlyStopping(monitor='val_loss', patience=3),
callbacks.ModelCheckpoint(filepath=checkpoint_path,
save_weights_only=True,
verbose=1)])
推荐答案
如果 filepath
不包含像 {epoch}
这样的格式选项,则 filepath
将被每个新的更好的模型覆盖.就您而言,这就是您无法在特定时期(例如时期 5)获得权重的原因.
If the filepath
doesn't contain formatting options like {epoch}
then filepath
will be overwritten by each new better model. In your case, that's why you can't get the weight at a specific epoch (e.g epoch 5).
然而,您在这里的选择是在训练期间在 ModelCheckpoint
回调中选择格式化选项.比如
Your option here, however, is to choose the formatting option in the ModelCheckpoint
callback during training time. Such as
tf.keras.callbacks.ModelCheckpoint(
filepath='model.{epoch:02d}-{val_loss:.4f}.h5',
save_freq='epoch', verbose=1, monitor='val_loss',
save_weights_only=True, save_best_only=False
)
这将以不同但方便的方式在每个时期保存模型权重(以 .h5
格式).此外,如果我们将 save_best_only
选择为 True
,它将以相同的方式保存最佳权重.
This will save the model weight (in .h5
format) at each epoch, in a different but convenient way. Additionally, if we choose save_best_only
to True
, it will save best weights in the same way.
代码示例
这是一个端到端工作示例供参考.我们将使用格式化选项以方便的方式保存每个时期的模型权重,我们将定义 filepath
参数如下:
Here is one end-to-end working example for reference. We will save model weights at each epoch in a convenient way with a formatting option that we will define the filepath
parameter as follows:
img = tf.random.normal([20, 32], 0, 1, tf.float32)
tar = np.random.randint(2, size=(20, 1))
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10, input_dim = 32, activation= 'relu'))
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))
callback_list = [
tf.keras.callbacks.ModelCheckpoint(
filepath='model.{epoch:02d}-{val_loss:.4f}.h5',
save_freq='epoch', verbose=1, monitor='val_loss',
save_weights_only=True, save_best_only=False
)
]
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(img, tar, epochs=5, verbose=2, validation_split=0.2,
callbacks=callback_list)
它将在每个时期保存模型权重.我会在本地磁盘中找到所有权重.
It will save the model weight at each epoch. And I will find every weight in my local disk.
# model.epoch_number_score.h5
model.01-0.8022.h5
model.02-0.8014.h5
model.03-0.8005.h5
model.04-0.7997.h5
model.05-0.7989.h5
但是,请注意,我使用了 save_best_only = False
,但是如果我们将其设置为 True
,那么您只能以相同的方式获得最佳权重.像这样:
However, note that I used save_best_only = False
, but If we set it to True
, you then only get the best weight in the same way. Something like this:
# model.epoch_number_score.h5
model.01-0.8022.h5
model.03-0.8005.h5
model.05-0.7989.h5
这篇关于从 Tensorflow Keras 检查点重新加载最佳权重的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!