Keras网络产生逆预测 [英] Keras network producing inverse predictions

问题描述

我有一个时间序列数据集，我正在尝试训练一个网络，使其过度拟合(显然，这只是第一步，然后我将解决过度拟合的问题).

I have a timeseries dataset and I am trying to train a network so that it overfits (obviously, that's just the first step, I will then battle the overfitting).

网络有两层: LSTM(32个神经元)和Dense(1个神经元，未激活)

The network has two layers: LSTM (32 neurons) and Dense (1 neuron, no activation)

训练/模型具有以下参数: epochs: 20，steps_per_epoch: 100，loss: "mse"，optimizer: "rmsprop".

Training/model has these parameters: epochs: 20, steps_per_epoch: 100, loss: "mse", optimizer: "rmsprop".

TimeseriesGenerator产生具有以下内容的输入序列:length: 1，sampling_rate: 1，batch_size: 1.

TimeseriesGenerator produces the input series with: length: 1, sampling_rate: 1, batch_size: 1.

我希望网络能够记住这么小的数据集(我尝试过更复杂的网络无济于事)，训练数据集的损失几乎为零.并非如此，当我在 training 集合上将结果可视化时是这样的:

I would expect the network would just memorize such a small dataset (I have tried even much more complicated network to no avail) and the loss on training dataset would be pretty much zero. It is not and when I visualize the results on the training set like this:

y_pred = model.predict_generator(gen)
plot_points = 40
epochs = range(1, plot_points + 1)
pred_points = numpy.resize(y_pred[:plot_points], (plot_points,))
target_points = gen.targets[:plot_points]
plt.plot(epochs, pred_points, 'b', label='Predictions')
plt.plot(epochs, target_points, 'r', label='Targets')
plt.legend()
plt.show()

我得到:

预测的幅度较小，但与目标恰好相反.顺便提一句.这是没有记住的，即使对于算法完全没有训练的测试数据集也是如此，似乎是我的网络没有记住数据集，而是学会了取反输入值并稍微缩小它，而不是记住数据集. 知道为什么会发生这种情况吗?看来，优化器不应该收敛到该解决方案上(损失相当大).

The predictions have somewhat smaller amplitude but are precisely inverse to the targets. Btw. this is not memorized, they are inversed even for the test dataset which the algorithm hasn't trained on at all.It appears that instead of memorizing the dataset, my network just learned to negate the input value and slightly scale it down. Any idea why this is happening? It doesn't seem like the solution the optimizer should have converged to (loss is pretty big).

编辑(我代码的一些相关部分):

EDIT (some relevant parts of my code):

train_gen = keras.preprocessing.sequence.TimeseriesGenerator(
        x,
        y,
        length=1,
        sampling_rate=1,
        batch_size=1,
        shuffle=False
    )

model = Sequential()
model.add(LSTM(32, input_shape=(1, 1), return_sequences=False))
model.add(Dense(1, input_shape=(1, 1)))

model.compile(
    loss="mse",
    optimizer="rmsprop",
    metrics=[keras.metrics.mean_squared_error]
)

history = model.fit_generator(
    train_gen,
    epochs=20,
    steps_per_epoch=100
)

编辑(不同的，随机生成的数据集):

EDIT (different, randomly generated dataset):

我不得不将LSTM神经元的数量增加到256，在以前的设置(32个神经元)下，蓝线几乎是平坦的.但是，随着增加，出现了相同的模式-幅度稍小的反向预测.

I had to increase number of LSTM neurons to 256, with the previous setting (32 neurons), the blue line was pretty much flat. However, with the increase the same pattern arises - inverse predictions with somewhat smaller amplitude.

编辑(目标移动+1):

EDIT (targets shifted by +1):

与预测相比，将目标移动1不会产生更好的拟合度.注意图中突出显示的部分不仅是交替的，而且在那里更明显.

Shifting the targets by one compared to predictions doesn't produce much better fit. Notice the highlighted parts where the graph isn't just alternating, it's more apparent there.

编辑(将长度增加到2 ... TimeseriesGenerator(length=2, ...)):

EDIT (increased length to 2 ... TimeseriesGenerator(length=2, ...)):

使用length=2时，预测会停止如此紧密地跟踪目标，但总体反演模式仍然存在.

With length=2 the predictions stop tracking the targets so closely but the overall pattern of inversion still stands.

推荐答案

您说您的网络刚刚学会对输入值求反并对其进行了小幅缩减".我不这么认为.您所看到的很可能只是网络性能不佳，而只是预测先前的值(但可以按您所说的进行扩展).这个问题是我一次又一次看到的. 这里是另一个例子，以及另一个.另外，请记住，将数据移动一个就很容易欺骗自己.您很有可能只是将糟糕的预测及时移回并得到了重叠.

You say that your network "just learned to negate the input value and slightly scale it down". I don't think so. It is very likely that all you are seeing is the network performing poorly, and just predicting the previous value (but scaled as you say). This issue is something I've seen again and again. Here is another example, and another, of this issue. Also, remember it is very easy to fool yourself by shifting the data by one. It is very likely you are simply shifting the poor prediction back in time and getting an overlap.

这篇关于Keras网络产生逆预测的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！