LSTM和Keras一起进行小批量培训和在线测试 [英] LSTM with Keras for mini-batch training and online testing

问题描述

我想在Keras中实现LSTM，以进行流时间序列预测-即在线运行，一次获取一个数据点. 这在这里得到了很好的解释，但正如人们所假设的那样，在线LSTM可能会非常慢.我想在小型批次上训练我的网络，并在线测试(运行预测).在Keras中执行此操作的最佳方法是什么?

I would like to implement an LSTM in Keras for streaming time-series prediction -- i.e., running online, getting one data point at a time. This is explained well here, but as one would assume, the training time for an online LSTM can be prohibitively slow. I would like to train my network on mini-batches, and test (run prediction) online. What is the best way to do this in Keras?

例如，小批量可以是在连续的时间步长处出现的1000个数据值([33, 34, 42, 33, 32, 33, 36, ... 24, 23])的序列.为了训练网络，我指定了形状为(900, 100, 1)的数组X，其中有900个长度为100的序列，以及形状为(900, 1)的数组y.例如，

For example, a mini-batch could be a sequence of 1000 data values ([33, 34, 42, 33, 32, 33, 36, ... 24, 23]) that occur at consecutive time steps. To train the network I've specified an array X of shape (900, 100, 1), where there are 900 sequences of length 100, and an array y of shape (900, 1). E.g.,

X[0] = [[33], [34], [42], [33], ...]]
X[1] = [[34], [42], [33], [32], ...]]
...
X[999] = [..., [24]]

y[999] = [23]

因此对于每个序列X[i]，都有一个对应的y[i]表示时间序列中的下一个值-我们要预测的值.

So for each sequence X[i], there is a corresponding y[i] that represents the next value in the time-series -- what we want to predict.

在测试中，我想预测下一个数据值1000到1999.我通过为1000到1999的每个步骤提供形状为(1, 100, 1)的数组来完成此操作，在该模型中，模型尝试在下一步预测值.

In test I want to predict the next data values 1000 to 1999. I do this by feeding an array of shape (1, 100, 1) for each step from 1000 to 1999, where the model tries to predict the value at the next step.

这是针对我的问题的推荐方法和设置吗?启用有状态性可能是纯在线实现的一种方式，但是在Keras中，这要求在培训和测试中保持一致的batch_input_shape，这对我的小批量培训然后进行在线测试的意图不起作用.还是有办法可以做到这一点?

Is this the recommended approach and setup for my problem? Enabling statefulness may be the way to go for a purely online implementation, but in Keras this requires a consistent batch_input_shape in training and testing, which would not work for my intent of training on mini-batches and then testing online. Or is there a way I can do this?

更新:尝试按照@nemo建议的方式实施网络

我通过博客文章，然后尝试将预测阶段实现为有状态网络.

I ran my own dataset on an example network from a blog post "Time Series Prediction with LSTM Recurrent Neural Networks in Python with Keras", and then tried implementing the prediction phase as a stateful network.

两者的模型构建和训练都相同:

The model building and training is the same for both:

# Create and fit the LSTM network
numberOfEpochs = 10
look_back = 30
model = Sequential()
model.add(LSTM(4, input_dim=1, input_length=look_back))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, nb_epoch=numberOfEpochs, batch_size=1, verbose=2)
# trainX.shape = (6883, 30, 1)
# trainY.shape = (6883,)
# testX.shape = (3375, 30, 1)
# testY.shape = (3375,)

批量预测通过以下方式完成:

Batch prediction is done with:

trainPredict = model.predict(trainX, batch_size=batch_size)
testPredict = model.predict(testX, batch_size=batch_size)

为尝试有状态预测阶段，我运行了与以前相同的模型设置和训练，但随后执行了以下操作:

To try a stateful prediction phase, I ran the same model setup and training as before, but then the following:

w = model.get_weights()
batch_size = 1
model = Sequential()
model.add(LSTM(4, batch_input_shape=(batch_size, look_back, 1), stateful=True))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
trainPredictions, testPredictions = [], []
for trainSample in trainX:
    trainPredictions.append(model.predict(trainSample.reshape((1,look_back,1)), batch_size=batch_size))
trainPredict = numpy.concatenate(trainPredictions).ravel()
for testSample in testX:
    testPredictions.append(model.predict(testSample.reshape((1,look_back,1)), batch_size=batch_size))
testPredict = numpy.concatenate(testPredictions).ravel()

要检查结果，下面的图以蓝色显示实际(规范化)数据，以绿色显示训练集的预测，以红色显示测试集的预测.

To inspect the results, the plots below show the actual (normalized) data in blue, the predictions on the training set in green, and the predictions on the test set in red.

第一个数字来自使用批处理预测，第二个数字来自有状态.有什么想法我做错了吗?

The first figure is from using batch prediction, and the second from stateful. Any ideas what I'm doing incorrectly?

推荐答案

如果我对您的理解正确，那么您在问您是否可以在训练后启用有状态性.这应该是可能的，是的.例如:

If I understand you correctly you are asking if you can enable statefulness after training. This should be possible, yes. For example:

net = Dense(1)(SimpleRNN(stateful=False)(input))
model = Model(input=input, output=net)

model.fit(...)

w = model.get_weights()
net = Dense(1)(SimpleRNN(stateful=True)(input))
model = Model(input=input, output=net)
model.set_weights(w)

之后，您可以以有状态的方式进行预测.

After that you can predict in a stateful way.

这篇关于LSTM和Keras一起进行小批量培训和在线测试的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！