如何创建简单的三层神经网络并使用监督学习进行教学? [英] How to create simple 3-layer neural network and teach it using supervised learning?
问题描述
基于 PyBrain的教程，我设法将以下代码组合在一起:
Based on PyBrain's tutorials I managed to knock together the following code:
#!/usr/bin/env python2
# coding: utf-8
from pybrain.structure import FeedForwardNetwork, LinearLayer, SigmoidLayer, FullConnection
from pybrain.datasets import SupervisedDataSet
from pybrain.supervised.trainers import BackpropTrainer
n = FeedForwardNetwork()
inLayer = LinearLayer(2)
hiddenLayer = SigmoidLayer(3)
outLayer = LinearLayer(1)
n.addInputModule(inLayer)
n.addModule(hiddenLayer)
n.addOutputModule(outLayer)
in_to_hidden = FullConnection(inLayer, hiddenLayer)
hidden_to_out = FullConnection(hiddenLayer, outLayer)
n.addConnection(in_to_hidden)
n.addConnection(hidden_to_out)
n.sortModules()
ds = SupervisedDataSet(2, 1)
ds.addSample((0, 0), (0,))
ds.addSample((0, 1), (1,))
ds.addSample((1, 0), (1,))
ds.addSample((1, 1), (0,))
trainer = BackpropTrainer(n, ds)
# trainer.train()
trainer.trainUntilConvergence()
print n.activate([0, 0])[0]
print n.activate([0, 1])[0]
print n.activate([1, 0])[0]
print n.activate([1, 1])[0]
本来应该学习XOR函数的，但是结果似乎很随机:
It's supposed to learn XOR function, but the results seem quite random:
0.208884929522
0.208884929522
0.168926515771
0.168926515771
0.459452834043
0.459452834043
0.424209192223
0.424209192223
或
0.84956138664
0.84956138664
0.888512762786
0.888512762786
0.564964077401
0.564964077401
0.611111147862
0.611111147862
推荐答案
There are four problems with your approach, all easy to identify after reading Neural Network FAQ:
-
为什么要使用偏差/阈值?:您应该添加一个偏置节点.偏见的缺乏使学习非常有限:网络代表的分离的超平面只能通过原点.借助bias节点，它可以自由移动并更好地拟合数据:
Why use a bias/threshold?: you should add a bias node. Lack of bias makes the learning very limited: the separating hyperplane represented by the network can only pass through the origin. With the bias node, it can move freely and fit the data better:
bias = BiasUnit()
n.addModule(bias)
bias_to_hidden = FullConnection(bias, hiddenLayer)
n.addConnection(bias_to_hidden)
为什么不将二进制输入编码为0和1?:所有样本都位于样本空间的一个象限中.将它们移动到原点周围:
Why not code binary inputs as 0 and 1?: all your samples lay in a single quadrant of the sample space. Move them to be scattered around the origin:
ds = SupervisedDataSet(2, 1)
ds.addSample((-1, -1), (0,))
ds.addSample((-1, 1), (1,))
ds.addSample((1, -1), (1,))
ds.addSample((1, 1), (0,))
(相应地在脚本末尾修正验证码.)
trainUntilConvergence方法使用验证工作，并且执行类似于提前停止方法.对于这么小的数据集，这没有任何意义.使用trainEpochs代替.对于这个问题，1000时代已足够网络学习:
trainUntilConvergence method works using validation, and does something that resembles the early stopping method. This doesn't make sense for such a small dataset. Use trainEpochs instead. 1000 epochs is more than enough for this problem for the network to learn:
trainer.trainEpochs(1000)
反向学习应使用哪种学习速率? :调整学习率参数.每当您使用神经网络时，便会执行此操作.在这种情况下，值0.1或什至0.2会极大地提高学习速度:
What learning rate should be used for backprop?: Tune the learning rate parameter. This is something you do every time you employ a neural network. In this case, the value 0.1 or even 0.2 dramatically increases the learning speed:
trainer = BackpropTrainer(n, dataset=ds, learningrate=0.1, verbose=True)
(请注意verbose=True参数.调整参数时，观察错误的行为至关重要.)
(Note the verbose=True parameter. Observing how the error behaves is essential when tuning parameters.)
使用这些修复程序，我可以获得具有给定数据集的给定网络的一致且正确的结果，并且误差小于1e-23.
With these fixes I get consistent, and correct results for the given network with the given dataset, and error less than 1e-23.
这篇关于如何创建简单的三层神经网络并使用监督学习进行教学?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
