递归神经网络外推 [英] extrapolation with recurrent neural network

问题描述

我编写了一个简单的递归神经网络(7个神经元，每个神经元最初都连接到所有神经元)，并使用遗传算法对其进行了训练，以学习复杂"的非线性函数，例如1/(1 + x ^ 2 ).作为训练集，我使用了[-5,5]范围内的20个值(我尝试使用20个以上的值，但结果并未发生太大变化).

网络可以很好地学习此范围，并且当给出此范围内其他点的示例时，它可以预测函数的值.但是，它无法正确外推并无法预测该函数的值在[-5,5]范围之外.是什么原因造成的，我该怎么做才能提高其推断能力?

谢谢！

解决方案

神经网络不是外推方法(无论是否重复)，这完全超出了它们的能力.它们用于将功能拟合到提供的数据上，它们完全可以自由地在填充有训练点的子空间之外构建模型.因此，从非严格意义上讲，应该将它们视为一种插值方法.

为了清楚起见，神经网络应该能够在训练样本所跨越的子空间内部进行泛化，而不能在训练空间之外进行泛化

仅在与训练样本保持一致的意义上训练神经网络，而外推则完全不同.来自"H.Lohninger:Teach/Me数据分析，Springer-Verlag，柏林-纽约-东京，1999年.ISBN3-540-14743-8"中的一个简单示例显示了NN在这种情况下的行为

所有这些网络都与训练数据一致，但是可以在此子空间之外进行任何操作.

您应该重新考虑问题的表述，如果可以将其表示为回归或分类问题，则可以使用NN，否则，您应该考虑一些完全不同的方法.

唯一可以通过某种方式纠正"训练集以外发生的事情的方法是:

• 在所需的子空间中添加人工训练点(但这只是增加了训练集，并且在新集合之外，网络的行为是随机的")
• 添加强大的正则化功能，这将迫使网络创建非常简单的模型，但是模型的复杂性将无法保证任何外推强度，因为两个完全相同的复杂性模型在-/+无穷大方面可能具有完全不同的限制.

将以上两个步骤结合起来可以帮助构建在某种程度上外推"的模型，但是如上所述，这并不是神经网络的目的.

I Wrote a simple recurrent neural network (7 neurons, each one is initially connected to all the neurons) and trained it using a genetic algorithm to learn "complicated", non-linear functions like 1/(1+x^2). As the training set, I used 20 values within the range [-5,5] (I tried to use more than 20 but the results were not changed dramatically).

The network can learn this range pretty well, and when given examples of other points within this range, it can predict the value of the function. However, it can not extrapolate correctly and predicting the values of the function outside the range [-5,5]. What are the reasons for that and what can I do to improve its extrapolation abilities?

Thanks!

解决方案

Neural networks are not extrapolation methods (no matter - recurrent or not), this is completely out of their capabilities. They are used to fit a function on the provided data, they are completely free to build model outside the subspace populated with training points. So in non very strict sense one should think about them as an interpolation method.

To make things clear, neural network should be capable of generalizing the function inside subspace spanned by the training samples, but not outside of it

Neural network is trained only in the sense of consistency with training samples, while extrapolation is something completely different. Simple example from "H.Lohninger: Teach/Me Data Analysis, Springer-Verlag, Berlin-New York-Tokyo, 1999. ISBN 3-540-14743-8" shows how NN behave in this context

All of these networks are consistent with training data, but can do anything outside of this subspace.

You should rather reconsider your problem's formulation, and if it can be expressed as a regression or classification problem then you can use NN, otherwise you should think about some completely different approach.

The only thing, which can be done to somehow "correct" what is happening outside the training set is to:

• add artificial training points in the desired subspace (but this simply grows the training set, and again - outside of this new set, network's behavious is "random")
• add strong regularization, which will force network to create very simple model, but model's complexity will not guarantee any extrapolation strength, as two model's of exactly the same complexity can have for example completely different limits in -/+ infinity.

Combining above two steps can help building model which to some extent "extrapolates", but this, as stated before, is not a purpose of a neural network.

这篇关于递归神经网络外推的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！