具有1个隐藏层的神经网络无法逼近简单乘法函数 [英] Can't approximate simple multiplication function in neural network with 1 hidden layer

问题描述

我只是想测试神经网络近似乘法功能(回归任务)的效果. 我正在使用Azure Machine Learning Studio.我有6500个样本，其中1个隐藏层 (我已经为每个隐藏层测试了5/30/100个神经元)，没有进行归一化处理.和默认参数 学习率-0.005，学习迭代次数-200，初始学习权重- 0.1， 动量-0 [描述] .我的准确度非常差，接近0. 同时，增强的决策森林回归显示出非常好的近似值.

I just wanted to test how good can neural network approximate multiplication function (regression task). I am using Azure Machine Learning Studio. I have 6500 samples, 1 hidden layer (I have tested 5 /30 /100 neurons per hidden layer), no normalization. And default parameters Learning rate - 0.005, Number of learning iterations - 200, The initial learning weigh - 0.1, The momentum - 0 [description]. I got extremely bad accuracy, close to 0. At the same time boosted Decision forest regression shows very good approximation.

我做错了什么?对于NN，这项任务应该非常容易.

What am I doing wrong? This task should be very easy for NN.

推荐答案

大乘法函数梯度会迫使网络几乎立即进入某种恐怖状态，其中所有隐藏节点的梯度均为零. 我们可以使用两种方法:

Big multiplication function gradient forces the net probably almost immediately into some horrifying state where all its hidden nodes have zero gradient. We can use two approaches:

1)乘以常数.我们只是在学习之前就准备好一切，然后再进行学习.

1) Devide by constant. We are just deviding everything before the learning and multiply after.

2)进行日志标准化.它将乘法相加:

2) Make log-normalization. It makes multiplication into addition:

m = x*y => ln(m) = ln(x) + ln(y).

这篇关于具有1个隐藏层的神经网络无法逼近简单乘法函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！