如何防止程序陷入局部最大值(前馈人工神经网络和遗传算法) [英] How can I prevent my program from getting stuck at a local maximum (Feed forward artificial neural network and genetic algorithm)
问题描述
我正在研究前馈人工神经网络(ffann)，它将以简单计算的形式获取输入并返回结果(充当袖珍计算器).结果不会是准确的.
利用遗传算法对权重进行人工训练.
I'm working on a feed forward artificial neural network (ffann) that will take input in form of a simple calculation and return the result (acting as a pocket calculator). The outcome wont be exact.
The artificial network is trained using genetic algorithm on the weights.
目前，我的程序停留在本地最大值:
Currently my program gets stuck at a local maximum at:
- 正确答案为5％-6％，错误幅度为1％
- 正确答案30％，错误余量10％
- 正确答案40％，错误率20％
- 正确答案占45％，错误率占30％
- 60％正确答案，错误率40％
我目前使用两种不同的遗传算法:
第一个是基本选择，从我的总体中随机选择两个，将一个最适合自己的人命名为获胜者，另一个则选择失败者.失败者从胜利者那里获得权重之一.
I currently use two different genetic algorithms:
The first is a basic selection, picking two random from my population, naming the one with best fitness the winner, and the other the loser. The loser receives one of the weights from the winner.
第二个是突变，选择中的失败者会根据产生的错误数量进行轻微修改. (适合性取决于正确答案和错误答案). 因此，如果网络输出很多错误，它将得到很大的修改，而如果它有很多正确的答案，我们将接近一个可接受的目标，并且修改会更小.
The second is mutation, where the loser from the selection receives a slight modification based on the amount of resulting errors. (the fitness is decided by correct answers and incorrect answers). So if the network outputs a lot of errors, it will receive a big modification, where as if it has many correct answers, we are close to a acceptable goal and the modification will be smaller.
那么问题来了:我有什么方法可以防止我的芬妮卡在局部最大值上?
我应该将当前的遗传算法修改为具有更多变量的更高级算法吗?
我应该创建其他突变还是交叉?
还是应该尝试将突变变量修改为更大或更小的值?
So to the question: What are ways I can prevent my ffann from getting stuck at local maxima?
Should I modify my current genetic algorithm to something more advanced with more variables?
Should I create additional mutation or crossover?
Or Should I maybe try and modify my mutation variables to something bigger/smaller?
这是一个大话题，因此，如果我错过了任何可能需要的信息，请发表评论
将突变数调整为更合适的值已经获得了更好的回答率，但远未获批准:
Tweaking the numbers of the mutation to a more suited value has gotten be a better answer rate but far from approved:
- 正确答案10％，错误幅度1％
- 33％的正确答案，错误率10％
- 43％的正确答案，错误率20％
- 65％的正确答案，错误率30％
- 正确答案73％，错误率40％
该网络目前是一个非常简单的3层结构，具有3个输入，在唯一的隐藏层有2个神经元，在输出层有一个神经元.
使用的激活函数为Tanh，其值介于-1和1之间.
选择类型交叉非常简单，如下所示:
The network is currently a very simple 3 layered structure with 3 inputs, 2 neurons in the only hidden layer, and a single neuron in the output layer.
The activation function used is Tanh, placing values in between -1 and 1.
The selection type crossover is very simple working like the following:
[a1, b1, c1, d1] // Selected as winner due to most correct answers
[a2, b2, c2, d2] // Loser
输家将最终从赢家那里获得一个价值，因为我认为(权重)数组中的位置对它的表现很重要，所以将价值直接下移.
The loser will end up receiving one of the values from the winner, moving the value straight down since I believe the position in the array (of weights) matters to how it performs.
突变非常简单，在失败者权重数组中为随机权重增加了一个很小的值(当前介于0.01到0.001之间)，有50/50的可能性为负值.
The mutation is very simple, adding a very small value (currently somewhere between about 0.01 and 0.001) to a random weight in the losers array of weights, with a 50/50 chance of being a negative value.
以下是一些训练数据的示例:
Here are a few examples of training data:
1, 8, -7 // the -7 represents + (1+8)
3, 7, -3 // -3 represents - (3-7)
7, 7, 3 // 3 represents * (7*7)
3, 8, 7 // 7 represents / (3/8)
推荐答案
在GA中使用适当的技术.一个有用的替代方法是小生境.考虑到总体总体的相似性，每个解决方案的得分(我认为是某种形式的二次误差)都会发生变化.这样可以保持种群内部的多样性，并避免过早收敛而陷入局部最优状态.
Use a niching techniche in the GA. A useful alternative is niching. The score of every solution (some form of quadratic error, I think) is changed in taking account similarity of the entire population. This maintains diversity inside the population and avoid premature convergence an traps into local optimum.
在这里看看:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi = 10.1.1.100.7342
这篇关于如何防止程序陷入局部最大值(前馈人工神经网络和遗传算法)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
