MLP神经网络无法学习 [英] MLP Neural Network Won't learn

问题描述

我已经遍历了所有代码，如果确实是问题所在，那么我不确定它是如何使我难以理解的.发布时间太长，因此我将告诉您我的问题，我正在寻找的解决方案，如果您有任何想法我可以搜索的其他内容，我将不胜感激！

I've gone through all my code and if it really is the problem then I'm not sure how it has eluded me. It's too long to post so I'll tell you my problem, what ive looked at for fixing and if you have any ideas what else I can search for I'd be very appreciative!

好吧，所以首先按照haykin的指示，将权重初始化为均值零，方差等于该神经元输入数量的1/平方根.

Ok, so firstly the weights are initialised with mean zero and variance equal to 1/ square root of the number of inputs for that neuron, as instructed by haykin.

我已经给它提供了一个简单的正弦波，首先要学习.隐藏层中的权重似乎收敛，从而为该层中的每个神经元提供相同的输出...这使得输出神经元提供了几乎固定的输出.

I've fed it a simple sine wave to learn on first. The weights in the hidden layer seem to converge to give the same output for each neuron within that layer... which makes the output neuron give a nearly fixed output.

那么，可能是什么原因?首先，我检查了网络学习率是否导致其陷入局部最小值并提高了最小值，并且还尝试了&没有动力.我发现它在某种程度上解决了这个问题，因为网络确实会产生正弦波.但是，不正确！ :(

So, what could be the cause? Firstly I checked if the network learning rate was causing it to get stuck in local minima and increased it, and also tried with & without momentum. I found it rectified the problem somewhat, as the network DOES produce the sine wave. However, not properly! :(

网络输出的幅度大约是从中心轴向上的高度的三分之一，并且不会低于该高度.看起来有点像您拾取了正弦波，将其压扁了三分之一，然后将其升高到轴上的最低峰.此外，顶峰都平坦...

The network output has an amplitude roughly a third the height from the centre axis up, and doesnt go below. It looks kind of like you've picked the sine wave up, squished it a third and raised it to sit with it's lowest peaks on the axis. Furthermore the top peaks are all flat...

自从我尝试更改网络拓扑后，如果我添加另一个隐藏的神经元(总数为3)，它突然只会给出固定的输出.

I since tried changing the network topology, whereby if I add another hidden neuron (total 3) it suddenly only gives a fixed output.

推荐答案

对于具有S型激活函数的神经网络，正弦波并不是一个容易解决的问题. 3个隐藏的神经元通常是不够的.看看此示例:要隐藏200个隐藏节点在[0，2 * pi]范围内近似正弦波.此示例中的激活功能为:隐藏层中的逻辑Sigmoid和输出层中的身份.您始终应该在输出层中使用身份进行回归.

A sine wave is not an easy problem for a neural network with sigmoid activation functions. 3 hidden neurons are usually not enough. Take a look at this example: there are 200 hidden nodes to approximate a sine wave within [0, 2*pi]. The activation functions in this example are: logistic sigmoid in the hidden layer and identity in the output layer. You always should use identity in the output layer for regression.

如果您没有获得良好的结果，那么降低学习率也可能是有益的.有时，由于学习率太大，梯度下降在误差函数的陡峭区域之间振荡.

When you do not get a good result it also might be benificial to decrease the learning rate. Sometimes gradient descent oscillates between steep regions of the error function because the learning rate is too big.

这篇关于MLP神经网络无法学习的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！