神经网络中的时间序列提前预测(N点提前预测)大规模迭代训练 [英] Time Series Ahead Prediction in Neural Network (N Point Ahead Prediction) Large Scale Iterative Training
问题描述
(N = 90)使用神经网络进行的点超前预测:
(N=90) Point ahead Prediction using Neural Network:
我试图预测提前3分钟，即提前180分.因为我将时间序列数据压缩为将每2个点的平均值作为1，所以我必须预测(N = 90)提前预测.
I am trying to predict 3 minutes ahead i.e. 180 points ahead. Because I compressed my time series data as taking the mean of every 2 points as one, I have to predict (N=90) step-ahead prediction.
我的时间序列数据以秒为单位.值在30-90之间.如下面的示例所示，它们通常从30移至90，然后从90移至30.
My time series data is given in seconds. The values are in between 30-90. They usually move from 30 to 90 and 90 to 30, as seen in the example below.
我的数据可以从以下地址访问: https://www.dropbox.com/s /uq4uix8067ti4i3/17HourTrace.mat
My data could be reach from: https://www.dropbox.com/s/uq4uix8067ti4i3/17HourTrace.mat
我在实施神经网络以预测未来的N点时遇到了麻烦.我唯一的功能是上次.我使用了Elman递归神经网络，也使用了newff.
I am having trouble in implementing neural network to predict N points ahead. My only feature is previous time. I used elman recurrent neural network and also newff.
在我的情况下，我需要预测90分.首先，我是如何手动分离输入数据和目标数据的: 例如:
In my scenario I need to predict 90 points ahead. First how I separated my input and target data manually: For Example:
data_in = [1,2,3,4,5,6,7,8,9,10]; //imagine 1:10 only defines the array index values.
N = 90; %predicted second ahead.
P(:, :) T(:) it could also be(2 theta time) P(:, :) T(:)
[1,2,3,4,5] [5+N] | [1,3,5,7,9] [9+N]
[2,3,4,5,6] [6+N] | [2,4,6,8,10] [10+N]
...
直到到达数据末尾
我在Elman递归神经网络中有100个输入点和90个输出点.什么是最有效的隐藏节点大小?
I have 100 input points and 90 output points in Elman recurrent neural networks. What could be the most efficient hidden node size?
input_layer_size = 90;
NodeNum1 =90;
net = newelm(threshold,[NodeNum1 ,prediction_ahead],{'tansig', 'purelin'});
net.trainParam.lr = 0.1;
net.trainParam.goal = 1e-3;
///在我开始训练时，我用卡尔曼滤波，将其归一化到[0,1]范围内，然后我重新整理了数据. 1)我将无法训练我的完整数据.首先，我尝试训练大约900,000的完整M数据，但并没有给我解决方案.
//At the beginning of my training I filter it with kalman, normalization into range of [0,1] and after that I shuffled the data. 1) I won't able to train my complete data. First I tried to train complete M data which is around 900,000, which didn't gave me a solution.
2)其次，我尝试了迭代训练.但是在每次迭代中，新添加的数据将与已经训练的数据合并.经过20,000个训练数据后，准确性开始下降.最初训练的1000个数据非常适合训练.但是，当我开始迭代合并新数据并继续进行训练后，训练精度会迅速下降90到20. 例如.
2) Secondly I tried iteratively training. But in each iteration the new added data is merged with already trained data. After 20,000 trained data the accuracy start to decreases. First trained 1000 data perfectly fits in training. But after when I start iterativelt merge the new data and continue to training, the training accuracy drops very rapidly 90 to 20. For example.
P = P_test(1:1000) T = T_test(1:1000) counter = 1;
while(1)
net = train(net,P,T, [], [] );%until it reaches to minimum error I train it.
[normTrainOutput] = sim(net,P, [], [] );
P = [ P P(counter*1000:counter*2000)]%iteratively new training portion of the data added.
counter = counter + 1; end
这种方法非常慢，并且在某一点之后不会给出任何好的结果.
This approach is very slow and after a point it won't give any good resuts.
我的第三种方法是迭代训练；它与以前的训练类似，但是在每次迭代中，我只训练1000个数据部分，而没有与以前训练的数据进行任何合并，例如，当我训练前1000个数据直到达到最小误差(> 95％)时准确性.训练后，当我对数据的后1000个部分执行相同操作时，它将覆盖权重，并且预测变量主要充当数据的最新训练部分.
My third approach was iteratively training; It was similar to previous training but in each iteration, I do only train the 1000 portion of the data, without do any merging with previous trained data.For example when I train first 1000 data until it gets to minimum error which has >95% accuracy. After it has been trained, when I have done the same for the second 1000 portion of the data;it overwrites the weight and the predictor mainly behave as the latest train portion of the data.
> P = P_test(1:1000) T = T_test(1:1000) counter = 1;
while(1)
> net = train(net,P,T, [], [] ); % I did also use adapt()
> [normTrainOutput] = sim(net,P, [], [] );
>
> P = [ P(counter*1000:counter*2000)]%iteratively only 1000 portion of the data is added.
> counter = counter + 1;
end
训练数据:该图是我训练后的训练集的快照，蓝线是原始时间序列，红线是在训练后的神经网络中的预测值. MSE约为50.
Trained DATA: This figure is snapshot from my trained training set, blue line is the original time series and red line is the predicted values with trained neural network. The MSE is around 50.
已测试的数据:在下图中，您可以使用神经网络查看我对测试数据的预测，该神经网络使用20,000个输入点进行了训练，同时使训练数据的MSE误差小于50.放.它能够捕获很少的模式，但大多数情况下我并没有给出真正的准确性.
Tested DATA: On the below picture, you can see my prediction for my testing data with the neural network, which is trained with 20,000 input points while keeping MSE error <50 for the training data set. It is able to catch few patterns but mostly I doesn't give the real good accuracy.
我无法成功使用任何一种方法.在每次迭代中，我还观察到alpha上的细微变化会完全覆盖已训练的数据，而更多地关注当前训练的数据部分. 我将无法提出解决方案.在迭代训练中，我应该将学习率保持在很小的水平并将时期数保持在尽可能小的水平.
I wasn't able to successes any of this approaches. In each iteration I also observe that slight change on the alpha completely overwrites to already trained data and more focus onto the currently trained data portion. I won't able to come up with a solution to this problem. In iterative training should I keep the learning rate small and number of epochs as small.
我找不到有效的方法来预测时间序列上的90点.为了预测未来N点，我应该做的任何建议，任何教程或信息链接.
And I couldn't find an efficient way to predict 90 points ahead in time series. Any suggestions that what should I do to do in order to predict N points ahead, any tutorial or link for information.
进行迭代训练的最佳方法是什么?在第二种方法中，当我达到15000的训练数据时，训练规模突然开始下降.我应该迭代地更改运行时的Alpha吗?
What is the best way for iterative training? On my second approach when I reach 15 000 of trained data, training size starts suddenly to drop. Iteratively should I change the alpha on run time?
==========
==========
任何建议或我做错的事情将不胜感激.
Any suggestion or the things I am doing wrong would be very appreciated.
我还实现了递归神经网络.但是在训练大数据方面我也遇到了同样的问题，是否有可能在(newelm)的递归神经网络中进行自适应学习(在线学习)?重量不会自动更新，我也看不到任何改善.
I also implemented recurrent neural network. But on training for large data I have faced with the same problems.Is it possible to do adaptive learning(online learning) in Recurrent Neural Networks for(newelm)? The weight won't update itself and I didn't see any improvement.
如果是，那怎么可能，我应该使用哪些功能?
If yes, how it is possible, which functions should I use?
net = newelm(threshold,[6, 8, 90],{'tansig','tansig', 'purelin'});
net.trainFcn = 'trains';
batch_size = 10;
while(1)
net = train(net,Pt(:, k:k+batch_size ) , Tt(:, k:k+batch_size) );
end
推荐答案
看看 Echo状态网络(ESN)或其他形式的储层计算.它们非常适合时间序列预测，非常易于使用和收敛.您完全不必担心网络的结构(中间层的每个神经元都有随机权重，不会改变).您仅学习输出权重.
Have a look at Echo State Networks (ESNs) or other forms of Reservoir Computing. They are perfect for time series prediction, very easy to use and converge fast. You don't need to worry about the structure of the network at all (every neuron in the mid-layer has random weights which do not change). You only learn the output weights.
如果我使用Echo State Networks正确理解了问题，我将训练网络以预测下一个点和前面的90点.只需在输出神经元中强迫所需的输出，然后执行岭回归以了解输出权重即可.
If I understood the problem correctly, with Echo State Networks, I would just train the network to predict the next point AND 90 points ahead. This can be done by simply forcing the desired output in the output neurons and then performing ridge regression to learn the output weights.
训练网络后运行网络时，在每一步 n 中，它将输出下一个点( n + 1 )，您将其反馈到网络作为输入(以继续迭代)，并领先90点( n + 90 )，您可以使用它进行任何操作-即:您也可以将其反馈给网络，以便它影响下一个输出.
When running the network after having trained it, at every step n, it would output the next point (n+1), which you would feed back to the network as input (to continue the iteration), and 90 points ahead (n+90), which you can do whatever you want with - i.e: you could also feed it back to the network so that it affects the next outputs.
很抱歉，答案不是很清楚.用简短的答案很难解释储层计算的工作原理，但是，如果您仅阅读链接中的文章，就会发现很容易理解这些原理.
Sorry if the answer is not very clear. It's hard to explain how reservoir computing works in a short answer, but if you just read the article in the link, you will find it very easy to understand the principles.
If you do decide to use ESNs, also read this paper to understand the most important property of ESNs and really know what you're doing.
根据系统的可预测性"，很难预测90分.例如，如果您要预测一个混沌系统，那么如果您预测得很远，则噪声会引入很大的误差.
Depending on how "predictable" your system is, predicting 90 points ahead may still be very difficult. For example if you're trying to predict a chaotic system, noise would introduce very large errors if you're predicting far ahead.
这篇关于神经网络中的时间序列提前预测(N点提前预测)大规模迭代训练的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
