SGD的训练准确性 [英] Training accuracy on SGD

问题描述

您如何计算SGD的训练准确性?您是否使用训练网络时使用的批处理数据进行计算?还是使用整个数据集? (针对每个批次优化迭代)

How do you compute for the training accuracy for SGD? Do you compute it using the batch data you trained your network with? Or using the entire dataset? (for each batch optimization iteration)

我尝试使用我训练网络时使用的批处理数据来计算每次迭代的训练精度.它几乎总是给我100％的训练精度(有时100％，90％，80％，总是10％的倍数，但是第一次迭代给了我100％).这是因为我正在针对该迭代训练的同一批数据计算准确性吗?还是我的模型过度拟合，它立即给了我100％的费用，但是验证准确性很低? (这是主要问题，如果可以接受，或者模型有问题)

I tried computing the training accuracy for each iteration using the batch data I trained my network with. And it almost always gives me 100% training accuracy (sometimes 100%, 90%, 80%, always multiples of 10%, but the very first iteration gave me 100%). Is this because I am computing the accuracy on the same batch data I trained it with for that iteration? Or is my model overfitting that it gave me 100% instantly, but the validation accuracy is low? (this is the main question here, if this is acceptable, or there is something wrong with the model)

这是我使用的超参数.

batch_size = 64
kernel_size = 60 #from 60 #optimal 2
depth = 15 #from 60 #optimal 15
num_hidden = 1000 #from 1000 #optimal 80

learning_rate = 0.0001
training_epochs = 8

total_batches = train_x.shape[0] // batch_size

推荐答案

在训练过程中计算批处理数据的训练精度是正确的.如果准确性数字始终是10％的倍数，则很可能是因为您的批次大小为10.例如，如果有8个训练输出与标签匹配，那么您的训练准确性将是80％.如果训练准确度数字上升和下降，则有两种主要可能性: 1.如果您在一个时期内多次打印出准确度数字，那是正常现象，尤其是在训练的初期，因为模型是针对不同的数据样本进行预测的； 2.如果您每个时期打印一次准确性，并且在训练的后期阶段看到训练准确性上升和下降，则意味着您的学习率太大.您需要在培训期间减少加班费. 如果这些操作不能解决您的问题，请提供更多详细信息，以便我们提供帮助.

Calculating the training accuracy on the batch data during the training process is correct. If the number of the accuracy is always multiple of 10%, then most likely it is because your batch size is 10. For example, if 8 of the training outputs match the labels, then your training accuracy will be 80%. If the training accuracy number goes up and down, there are two main possibilities: 1. If you print out the accuracy numbers multiple time over one epoch, it is normal, especially at the early stage of training, because the model is predicting over different data samples; 2. If you print out the accuracy once each epoch, and if you see the training accuracy goes up and down during the later stage of the training, that means your learning rate is too big. You need to decease that overtime during the training. If these do not answer your question, please provider more details so that we can help.

这篇关于SGD的训练准确性的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！