在反向传播中使用 sigmoid 函数计算误差 [英] Calculate the error using a sigmoid function in backpropagation

问题描述

我有一个关于反向传播的快速问题.我正在查看以下内容:

I have a quick question regarding backpropagation. I am looking at the following:

http://www4.rgu.ac.uk/文件/第3%20-%20bp.pdf

在本文中，它说计算神经元的误差为

In this paper, it says to calculate the error of the neuron as

Error = Output(i) * (1 - Output(i)) * (Target(i) - Output(i))

Error = Output(i) * (1 - Output(i)) * (Target(i) - Output(i))

我把公式中我不明白的部分用粗体表示.在论文中，它说由于 sigmoid 函数需要 Output(i) * (1 - Output(i)) 项 - 但我仍然不明白为什么这会很必要.

I have put the part of the equation that I don't understand in bold. In the paper, it says that the Output(i) * (1 - Output(i)) term is needed because of the sigmoid function - but I still don't understand why this would be nessecary.

使用会有什么问题

Error = abs(Output(i) - Target(i))

?

误差函数是否与神经元激活/传递函数无关?

Is the error function regardless of the neuron activation/transfer function?

推荐答案

你需要这个的原因是你正在计算关于神经元输入的误差函数的导数.

The reason you need this is that you are calculating the derivative of the error function with respect to the neuron's inputs.

当你通过链式法则求导数时，你需要乘以神经元的激活函数(恰好是一个sigmoid)的导数

When you take the derivative via the chain rule, you need to multiply by the derivative of the neuron's activation function (which happens to be a sigmoid)

这是重要的数学.

通过链式法则计算神经元输入误差的导数:

Calculate the derivative of the error on the neuron's inputs via the chain rule:

E = -(target - output)^2

dE/dinput = dE/doutput * doutput/dinput

计算doutput/dinput:

Work out doutput/dinput:

output = sigmoid (input)

doutput/dinput = output * (1 - output)    (derivative of sigmoid function)

因此:

dE/dinput = 2 * (target - output) * output * (1 - output)

这篇关于在反向传播中使用 sigmoid 函数计算误差的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！