自我实现的神经网络奇异收敛 [英] Self-implemented neural-network strange convergance

问题描述

我目前遇到的问题是，使用Java的自实现神经网络已经有好几天了.我的网络有两个输入神经元，分别对应给定灰度图像中像素的x和y坐标，一个输出代表像素的色调.我正在使用的学习算法是RPROP.我遇到的问题是，在将整个训练数据集馈入网络的无数次迭代之后，它收敛到了一个与预期值相距很远的点，并且在分析了聚合网络的权重结构之后，我可以看到所有神经元在隐藏层中，它们具有完全相同的输入权重集.这与隐藏层中神经元的数量无关.是因为我用相同的值初始化网络中的所有权重吗?我尝试将权重随机化，但这并没有使事情变得更好.我在输出层以外的所有层中都使用了S型(tanh)激活函数.我不知道我是否在实现中犯了错误，或者是否误解了神经网络学习过程的数学描述的某些部分.有人知道什么可能导致这种奇怪的行为吗?

I am currently having a problem that I haven't been able to figure out for a couple of days with a self-implemented neural network in Java. My network has two input neurons corresponding to x and y coordinates of a pixel in a given greyscale image and one output representing the tone of the pixel. The learning algorithm I'm using is RPROP. The problem I am having is that after numerous iterations of feeding the network with the whole training data set, it converges to a point very far away from the one expected and after analysing the weight structure of the converged network, I could see that all neurons in the hidden layer had the exact same set of input weights. This happens independently of the number of neurons in the hidden layer. Is it caused by the fact that I'm initializing all weights in the network with the same value? I tried randomizing the weights but it didn't make things any better. I am using sigmoidal (tanh) activation functions in all the layers except the output layer. I don't know if I made a bug in implementation or if I misunderstood some part of mathematical description of neural network learning process. Does anyone know what might cause such strange behaviour?

推荐答案

尝试使用网络正在执行的打印输出进行调试.如果单位数量少，则可以打印激活，增量，权重和权重的更改.如果您有很多商品，可以尝试打印出其绝对值的平均值.

Try to debug with printouts of what the network is doing. If you have a small number of units, you can print out the activations, deltas, weights and changes to the weights. If you have a lot - you can try and print out the average value of their absolute value.

这可能会导致您遇到问题.

This may lead you to the problem.

还-您是否尝试过使用学习速率/动量?

Also - have you tried playing with the learning rate/momentum?

这篇关于自我实现的神经网络奇异收敛的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！