多层感知器 - XOR反向传播 [英] Multi-layer perceptron - XOR backpropagation
问题描述
所以我正在尝试为XOR实现MLP。目前我没有得到任何令人满意的结果。我用java编写它，目前我确信反向传播是正确的，前进也是如此，但我不知道为什么它不起作用。 delta计算是我遇到的问题，但我想我现在明白了。我希望有人可以帮助我。我已经将学习率从0.1设置为0.7并且没有运气。
我已经使用输入测试保持简单我使用了2个输入：00,01,10,11。结果我得到：
0.03046590405786709
0.9159156257800326
0.8664182057539402
0.7590399051848435
他们应该是0110，但网络没有正常学习。我测试了前进的结果，它们是正确的，所以我不太确定如何检查的是反向传播结果。 delta结果似乎也没问题。如果我的算法理解错误，请指导我。
我尝试过：
So I am trying to implement an MLP for XOR. Currently I am not getting any satisfying results. I coded it in java and currently I'm convinced that the backpropagation is correct and so is the forward but I don't get why it doesn't work. The delta calculation is what I had the mot problem with but I think I understand it now. I hope someone could help me. I have set the learning rate from 0.1 to 0.7 and no luck.
I already tested with the inputs keeping it simple I used 2 inputs: 00, 01,10,11. The results I'm getting:
0.03046590405786709
0.9159156257800326
0.8664182057539402
0.7590399051848435
They should be 0110 but the network isn't learning properly. I tested the results from the forward and they were correct so the only one I'm not quite sure how to check are the back propagation results. The delta results seem to be okay as well. Please guide me if my algorithm understanding is wrong.
What I have tried:
public double[] forward(ArrayList<Double> inputs){
// Loop through the hidden layers
for(int i = 0; i < NH; i++){
Hidden[i] = 0.0;
for(int j = 0; j < NI; j++){
Hidden[i] += (input[j] * W1[j][i]);
}
Hidden[i] +=bias;
//Calculate activation
Hidden[i] = 1/ (1 + Math.exp(-Hidden[i]));
}
// Outer layer - output calculation
for(int i = 0; i < NO; i++){
Output[i] = 0.0;
for(int j = 0; j < NH; j++){
Output[i] += (Hidden[j] * W2[j][i]);
}
//add bias set to 1
Output[i] +=bias;
//Calculate activation
Output[i] = 1/ (1 + Math.exp(-Output[i]));
}
return Output;
}
public double backwards(ArrayList<Double> targetOutputs){
double error = 0;
for(int i = 0; i < NO; i++){
Z2[i] = Output[i]*(1.0-Output[i]) * (targetOutputs.get(i) - Output[i]);
}
for(int i=0; i< NH; i++) {
error = 0.0;
for(int j=0; j< NO; j++){
error += W2[i][j] * Z2[j];
}
Z1[i] = Hidden[i] * (1.0-Hidden[i]) * error;
}
//calculate the delta weight change
for(int i=0; i< NH; i++) {
for(int j=0; j< NI; j++) {
dW1[j][i] += Z1[i] * input[j] + LR*dW1[j][i];
}
}
for(int i=0; i< NO; i++){
for(int j=0; j< NH; j++){
dW2[j][i] += Z2[i] * Hidden[j] + LR*dW2[j][i];
}
}
return Z2[0];
}
public void updateWeights(double LR){
for(int i= 0; i< NO; i++){
for(int j=0; j< NH; j++){
W2[j][i] += dW2[j][i] ;
dW2[j][i] = 0;
}
}
for(int i=0; i< NH; i++){
for(int j=0; j< NI; j++){
W1[j][i] += dW1[j][i];
dW1[j][i] = 0;
}
}
}
推荐答案
我确信反向传播是正确的，前向
I'm convinced that the backpropagation is correct and so is the forward
被说服是不够的！你必须确保事情是正确的。
尝试使用真实单元测试或手动进行单元测试：获取样本数据集，运行其中的函数，检查结果是否符合预期。
使用调试器来查看奇怪的附加位置也是一个好主意。
调试器允许你逐行跟踪执行，检查变量，你会发现有一点它会停止你所期望的。
调试器 - 维基百科，免费的百科全书 [ ^ ]
掌握Visual Studio 2010中的调试 - 初学者指南 [ ^ ]
http://docs.oracle.com/javase/7/ docs / technotes / tools / windows / jdb.html [ ^ ]
https://www.jetbrains.com/idea/help/debugging-your-first-java-application.html [ ^ ]
being convinced is not enough ! you must ensure that things are correct.
Try unit testing, either with real unit testing or manually: get a sample dataset, run the function on it, check that the results match the expectations.
It is also a good idea to use the debugger to see where something strange append.
The debugger allow you to follow the execution line by line, inspect variables and you will see that there is a point where it stop doing what you expect.
Debugger - Wikipedia, the free encyclopedia[^]
Mastering Debugging in Visual Studio 2010 - A Beginner's Guide[^]
http://docs.oracle.com/javase/7/docs/technotes/tools/windows/jdb.html[^]
https://www.jetbrains.com/idea/help/debugging-your-first-java-application.html[^]
这篇关于多层感知器 - XOR反向传播的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
