凯拉斯(Keras)提前停止 [英] Keras Early Stopping

问题描述

我正在使用Keras为我的项目训练神经网络. Keras提供了提前停止的功能.我是否知道应该使用哪些参数来避免神经网络因使用早期停止而过度拟合?

I'm training neural network for my project using Keras. Keras has provided a function for early stopping. May I know what parameters should be observed to avoid my neural network from overfitting by using early stopping?

推荐答案

一旦您的损失开始增加(或换句话说，验证准确性开始降低)，早期停止基本上就是停止训练.根据文档，其用法如下；

Early stopping is basically stopping the training once your loss starts to increase (or in other words validation accuracy starts to decrease). According to documents it is used as follows;

keras.callbacks.EarlyStopping(monitor='val_loss',
                              min_delta=0,
                              patience=0,
                              verbose=0, mode='auto')

值取决于您的实现(问题，批处理大小等)，但通常是为了防止过度使用；

Values depends on your implementation (problem, batch size etc...) but generally to prevent overfitting I would use;

1. 监控验证损失(需要使用交叉 验证或至少训练/测试集)，方法是设置monitor 'val_loss'的参数.
2. min_delta是一个阈值，用于确定是否将某个时期的损失量化为 改善与否.如果损耗差低于min_delta，则将其量化 没有任何改善.最好将其保留为0，因为我们对此感兴趣 当损失变得更严重时.
3. patience参数表示一旦损失开始增加(停止改善)，则停止之前的时期数. 如果您使用的是非常小的批次，这取决于您的实现方式 或大学习率您的损失 zig-zag (准确度会更高)，因此最好设置 大patience参数.如果您使用大批量和小批量 学习率，您的损失会更加顺畅，因此您可以使用 较小的patience参数.无论哪种方式，我都将其保留为2，所以我会 给模型更多机会.
4. verbose决定要打印的内容，将其保留为默认值(0).
5. mode参数取决于所监视数量的方向 具有(应该减少还是增加)，因为我们监视损失，所以可以使用min.但我们离开喀拉拉邦 为我们处理并设置为auto

1. Monitor the validation loss (need to use cross validation or at least train/test sets) by setting the monitor argument to 'val_loss'.
2. min_delta is a threshold to whether quantify a loss at some epoch as improvement or not. If the difference of loss is below min_delta, it is quantified as no improvement. Better to leave it as 0 since we're interested in when loss becomes worse.
3. patience argument represents the number of epochs before stopping once your loss starts to increase (stops improving). This depends on your implementation, if you use very small batches or a large learning rate your loss zig-zag (accuracy will be more noisy) so better set a large patience argument. If you use large batches and a small learning rate your loss will be smoother so you can use a smaller patience argument. Either way I'll leave it as 2 so I would give the model more chance.
4. verbose decides what to print, leave it at default (0).
5. mode argument depends on what direction your monitored quantity has (is it supposed to be decreasing or increasing), since we monitor the loss, we can use min. But let's leave keras handle that for us and set that to auto

因此，我将使用类似的方法并通过绘制有无提前停止的错误损失来进行实验.

So I would use something like this and experiment by plotting the error loss with and without early stopping.

keras.callbacks.EarlyStopping(monitor='val_loss',
                              min_delta=0,
                              patience=2,
                              verbose=0, mode='auto')

---

为避免对回调的工作方式可能产生歧义，我将尽力解释更多信息.在模型上调用fit(... callbacks=[es])后，Keras会调用给定的回调对象预定的函数.这些功能可以称为on_train_begin，on_train_end，on_epoch_begin，on_epoch_end和on_batch_begin，on_batch_end.在每个时期结束时调用提前停止回调，将最佳监视值与当前监视值进行比较，并在条件满足时停止(观察最佳监视值以来已经过去了多少个时期，这不仅仅是耐心的论点，两者之间的差异最后一个值大于min_delta等.).

---

For possible ambiguity on how callbacks work, I'll try to explain more. Once you call fit(... callbacks=[es]) on your model, Keras calls given callback objects predetermined functions. These functions can be called on_train_begin, on_train_end, on_epoch_begin, on_epoch_end and on_batch_begin, on_batch_end. Early stopping callback is called on every epoch end, compares the best monitored value with the current one and stops if conditions are met (how many epochs have past since the observation of the best monitored value and is it more than patience argument, the difference between last value is bigger than min_delta etc..).

正如@BrentFaust在评论中指出的那样，模型的训练将继续进行，直到满足Early Stopping条件或满足fit()中的epochs参数(默认值= 10)为止.设置Early Stopping回调不会使模型超出其epochs参数进行训练.因此，使用更大的epochs值调用fit()函数将从Early Stopping回调中受益更多.

As pointed by @BrentFaust in comments, model's training will continue until either Early Stopping conditions are met or epochs parameter (default=10) in fit() is satisfied. Setting an Early Stopping callback will not make the model to train beyond its epochs parameter. So calling fit() function with a larger epochs value would benefit more from Early Stopping callback.

这篇关于凯拉斯(Keras)提前停止的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！