如何使用深度神经网络提高验证准确性？ [英] How to increase validation accuracy with deep neural net?
问题描述
我正在尝试使用13000个训练图像和3000个验证图像构建11类图像分类器。我正在使用使用mxnet进行训练的深度神经网络。训练准确性正在提高，并达到80％以上，但验证准确性在54-57％的范围内，并且没有增加。
这可能是什么问题？我应该增加图像数量吗？
I am trying to build a 11 class image classifier with 13000 training images and 3000 validation images. I am using deep neural network which is being trained using mxnet. Training accuracy is increasing and reached above 80% but validation accuracy is coming in range of 54-57% and its not increasing. What can be the issue here? Should I increase the no of images?
推荐答案
这里的问题是您的网络在某个时候停止学习有用的常规功能，并开始适应您的训练集的特点（结果过度拟合）。您想强迫网络以继续学习有用的功能，并且这里的选择很少：
The issue here is that your network stop learning useful general features at some point and start adapting to peculiarities of your training set (overfitting it in result). You want to 'force' your network to keep learning useful features and you have few options here:
- 使用权重正则化。它试图使权重保持较低，这通常会导致更好的概括性。使用不同的正则化系数进行实验。尝试使用0.1、0.01、0.001，看看它们对准确性有什么影响。
- 破坏您的输入（例如，将一些像素随机替换为黑色或白色）。这样，您可以从输入中删除信息，并强制网络使用重要的常规功能。使用噪声系数进行试验，该系数确定应破坏多少输入。研究表明，任何介于15％-45％范围内的效果都很好。
- 扩展训练集。由于您要处理图片，因此可以通过旋转/缩放等方式扩展现有图片（按照建议）。您还可以尝试对图像进行预处理（例如，将其映射为黑白，灰度等），但是此技术的效果取决于您的确切图像和类）
- 使用降噪标准对您的图层进行预训练。在这里，您可以分别对网络的每一层进行预训练，然后再对整个网络进行微调。预训练迫使各层采用重要的常规功能，这些功能对于重建输入信号非常有用。例如，研究一下自动编码器（它们过去曾应用于图像分类）。
- 具有网络体系结构的实验。您的网络可能没有足够的学习能力。实验不同的神经元类型，层数和隐藏神经元数。确保尝试压缩架构（神经元少于输入）和稀疏架构（神经元比输入更多）。
- Use weight regularization. It tries to keep weights low which very often leads to better generalization. Experiment with different regularization coefficients. Try 0.1, 0.01, 0.001 and see what impact they have on accuracy.
- Corrupt your input (e.g., randomly substitute some pixels with black or white). This way you remove information from your input and 'force' the network to pick up on important general features. Experiment with noising coefficients which determines how much of your input should be corrupted. Research shows that anything in the range of 15% - 45% works well.
- Expand your training set. Since you're dealing with images you can expand your set by rotating / scaling etc. your existing images (as suggested). You could also experiment with pre-processing your images (e.g., mapping them to black and white, grayscale etc. but the effectiveness of this technique will depend on your exact images and classes)
- Pre-train your layers with denoising critera. Here you pre-train each layer of your network individually before fine tuning the entire network. Pre-training 'forces' layers to pick up on important general features that are useful for reconstructing the input signal. Look into auto-encoders for example (they've been applied to image classification in the past).
- Experiment with network architecture. Your network might not have sufficient learning capacity. Experiment with different neuron types, number of layers, and number of hidden neurons. Make sure to try compressing architectures (less neurons than inputs) and sparse architectures (more neurons than inputs).
不幸的是，训练过程泛化得很好的网络需要大量的实验，并且需要一点人工监督才能对参数空间进行几乎是蛮力的探索（您会看到许多采用这种方法的研究工作）。最好为每个参数尝试3-5个值，看看它是否将您引向某个地方。
Unfortunately the process of training network that generalizes well involves a lot of experimentation and almost brute force exploration of parameter space with a bit of human supervision (you'll see many research works employing this approach). It's good to try 3-5 values for each parameter and see if it leads you somewhere.
当您尝试绘制精度/成本/ f1作为迭代次数的函数时并查看其行为。通常，您会注意到测试仪的准确性达到峰值，然后连续下降。因此，除了良好的体系结构，正则化，损坏等之外，您还在寻找产生最佳结果的大量迭代。
When you experiment plot accuracy / cost / f1 as a function of number of iterations and see how it behaves. Often you'll notice a peak in accuracy for your test set, and after that a continuous drop. So apart from good architecture, regularization, corruption etc. you're also looking for a good number of iterations that yields best results.
另一个提示：请确保每次培训时代随机化图像的顺序。
One more hint: make sure each training epochs randomize the order of images.
这篇关于如何使用深度神经网络提高验证准确性？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
