当深度学习神经网络在验证准确性方面停止提高时，下一步该怎么做? [英] What to do next when Deep Learning neural network stop improving in term of validation accuracy?

问题描述

我遇到了这个问题，我的模型仅在大约20或30个历元之后才非常快地收敛我的数据集包含7000个样本，我的神经网络具有3个隐层，每个隐层有18个神经元，并且批次归一化的落差为0.2.

I was running into this issue where my model converge very fast only after about 20 or 30 epoch My data set contain 7000 sample and my neural network has 3 hidden layer, each with 18 neurons and batch normalization with drop out 0.2.

我的任务是一个多标签分类，其中我的标签为[0 0 1]，[0 1 0]，[1 0 0]和[0 0 0]

My task is a multi label classification where my label are [0 0 1] , [0 1 0], [1 0 0] and [0 0 0]

num_neuron = 18
model = Sequential()
model.add(Dense(num_neuron, input_shape=(input_size,), activation='elu'))
model.add(Dropout(0.2))
model.add(keras.layers.BatchNormalization())

model.add(Dense(num_neuron, activation='elu'))
model.add(Dropout(0.2))
model.add(keras.layers.BatchNormalization())

model.add(Dense(num_neuron/3, activation='elu'))
model.add(Dropout(0.2))
model.add(keras.layers.BatchNormalization())

model.add(Dense(3, activation='sigmoid'))
model.compile(loss='binary_crossentropy',
              optimizer='nadam',
              metrics=['accuracy'])
history = model.fit(X_train, Y_train,batch_size=512 ,epochs=1000,
                    validation_data=(X_test, Y_test), verbose=2)

我想知道是否有什么办法可以做得更好，因为即使我设定了1000个时期，也不会真正改变

I was wondering if there is anything I can do to improve more because even after I set put for 1000 epoch, nothing would really change

推荐答案

这是神经网络训练中的预期行为:过一会儿，据说训练过程已经收敛，这意味着进一步的训练不会完成.t会导致任何进一步的进展(实际上，训练时间过长甚至可能损害模型的泛化能力，因为它可能导致对训练集的过度拟合.因此已尽早停止以解决此问题.)

This is the expected behaviour in the training of a neural network: after a while, the training process is said to have converged, which means that further training doesn't lead to any further progress (in fact, training for too long may even hurt the model's generalization capacity, since it may lead to overfitting to the training set. Early stopping was created to tackle this issue).

在您的情况下，由于培训已经收敛并且培训和验证损失都没有减少，因此可以肯定地说，通过此特定的培训过程和特定的方法，您已针对此特定任务实现了最高的准确性模型架构(具有18个神经元的3个隐藏层).

In your case, since training has already converged and neither the training nor the validation loss are decreasing anymore, it is safe to say that you have achieved the highest possible accuracy for this specific task, with this specific training procedure and this specific model architecture (3 hidden layers with 18 neurons).

但是，仍然可以通过试验这些属性来进行改进.在您的情况下，很难说，因为我不知道您要训练的任务，但是由于您的训练损失与验证损失几乎相同，因此您的模型可能不合适，这意味着您可能会得到如果您使用功能更强大的模型(每层具有更多神经元或更多层)或降低归一化效果，则效果更好.

It is still possible to make improvements, however, by experimenting with these properties. In your case, it is hard to say since I don't know the task that you're training for, but since your training loss is almost the same as the validation loss, your model is probably underfitting, meaning that it will likely get better if you use a more capable model (with more neurons per layer or more layers) or decrease normalization.

这篇关于当深度学习神经网络在验证准确性方面停止提高时，下一步该怎么做?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！