哪些参数应该用于提前停止? [英] Which parameters should be used for early stopping?

问题描述

我正在使用 Keras 为我的项目训练神经网络.Keras 提供了提前停止的功能.我可以知道应该观察哪些参数才能通过使用提前停止来避免我的神经网络过度拟合吗?

I'm training a neural network for my project using Keras. Keras has provided a function for early stopping. May I know what parameters should be observed to avoid my neural network from overfitting by using early stopping?

推荐答案

提前停止基本上是在您的损失开始增加时停止训练(或者换句话说，验证准确性开始下降).根据文档，其用法如下；

Early stopping is basically stopping the training once your loss starts to increase (or in other words validation accuracy starts to decrease). According to documents it is used as follows;

keras.callbacks.EarlyStopping(monitor='val_loss',
                              min_delta=0,
                              patience=0,
                              verbose=0, mode='auto')

值取决于您的实现(问题、批量大小等)，但通常是为了防止过度拟合，我会使用；

Values depends on your implementation (problem, batch size etc...) but generally to prevent overfitting I would use;

1. 监控验证损失(需要使用cross验证或至少训练/测试集)通过设置 monitor'val_loss' 的参数.
2. min_delta 是是否将某个时期的损失量化为的阈值改善与否.如果损失的差异在min_delta以下，则量化因为没有改善.最好将其保留为 0，因为我们对当损失变得更糟时.
3. patience 参数表示损失开始增加(停止改善)后停止之前的 epoch 数.这取决于您的实施，如果您使用非常小批量或者大学习率你的损失zig-zag(精度会更嘈杂)所以最好设置一个大耐心参数.如果您使用大批量和小批量学习率你的损失会更平滑，所以你可以使用较小的 patience 参数.无论哪种方式，我都会将其保留为 2 所以我会给模型更多机会.
4. verbose 决定打印什么，保留默认值 (0).
5. mode 参数取决于您监控的数量的方向有(它应该是减少还是增加)，因为我们监控损失，我们可以使用min.但是让我们离开 keras为我们处理并将其设置为 auto

1. Monitor the validation loss (need to use cross validation or at least train/test sets) by setting the monitor argument to 'val_loss'.
2. min_delta is a threshold to whether quantify a loss at some epoch as improvement or not. If the difference of loss is below min_delta, it is quantified as no improvement. Better to leave it as 0 since we're interested in when loss becomes worse.
3. patience argument represents the number of epochs before stopping once your loss starts to increase (stops improving). This depends on your implementation, if you use very small batches or a large learning rate your loss zig-zag (accuracy will be more noisy) so better set a large patience argument. If you use large batches and a small learning rate your loss will be smoother so you can use a smaller patience argument. Either way I'll leave it as 2 so I would give the model more chance.
4. verbose decides what to print, leave it at default (0).
5. mode argument depends on what direction your monitored quantity has (is it supposed to be decreasing or increasing), since we monitor the loss, we can use min. But let's leave keras handle that for us and set that to auto

所以我会使用类似的方法并通过绘制有和没有提前停止的错误损失来进行实验.

So I would use something like this and experiment by plotting the error loss with and without early stopping.

keras.callbacks.EarlyStopping(monitor='val_loss',
                              min_delta=0,
                              patience=2,
                              verbose=0, mode='auto')

<小时>

对于回调如何工作可能存在的歧义，我将尝试解释更多.一旦您在模型上调用 fit(... callbacks=[es])，Keras 就会调用给定的回调对象预定函数.这些函数可以调用on_train_begin、on_train_end、on_epoch_begin、on_epoch_end 和on_batch_begin，on_batch_end.提前停止回调在每个 epoch 结束时调用，将最佳监控值与当前值进行比较，如果满足条件则停止(自观察到最佳监控值以来已经过去了多少个 epoch，这是否不仅仅是耐心参数，之间的差异最后一个值大于 min_delta 等.).

---

For possible ambiguity on how callbacks work, I'll try to explain more. Once you call fit(... callbacks=[es]) on your model, Keras calls given callback objects predetermined functions. These functions can be called on_train_begin, on_train_end, on_epoch_begin, on_epoch_end and on_batch_begin, on_batch_end. Early stopping callback is called on every epoch end, compares the best monitored value with the current one and stops if conditions are met (how many epochs have past since the observation of the best monitored value and is it more than patience argument, the difference between last value is bigger than min_delta etc..).

正如@BrentFaust 在评论中指出的那样，模型的训练将继续，直到满足提前停止条件或满足 fit() 中的 epochs 参数(默认值 = 10).设置提前停止回调不会使模型训练超出其 epochs 参数.因此，使用更大的 epochs 值调用 fit() 函数会从 Early Stopping 回调中受益更多.

As pointed by @BrentFaust in comments, model's training will continue until either Early Stopping conditions are met or epochs parameter (default=10) in fit() is satisfied. Setting an Early Stopping callback will not make the model to train beyond its epochs parameter. So calling fit() function with a larger epochs value would benefit more from Early Stopping callback.

这篇关于哪些参数应该用于提前停止?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！