具有不同时间步长的用于 RNN 的 Keras 掩码 [英] Keras Masking for RNN with Varying Time Steps

问题描述

我正在尝试使用具有不同时间长度的序列在 Keras 中拟合 RNN.我的数据位于格式 (sample, time, feature) = (20631, max_time, 24) 的 Numpy 数组中，其中 max_time 在运行时确定为具有最多时间戳的样本可用的时间步长.我已经用 0 填充了每个时间序列的开头，显然最长的除外.

I'm trying to fit an RNN in Keras using sequences that have varying time lengths. My data is in a Numpy array with format (sample, time, feature) = (20631, max_time, 24) where max_time is determined at run-time as the number of time steps available for the sample with the most time stamps. I've padded the beginning of each time series with 0, except for the longest one, obviously.

我最初是这样定义我的模型的...

I've initially defined my model like so...

model = Sequential()
model.add(Masking(mask_value=0., input_shape=(max_time, 24)))
model.add(LSTM(100, input_dim=24))
model.add(Dense(2))
model.add(Activation(activate))
model.compile(loss=weibull_loglik_discrete, optimizer=RMSprop(lr=.01))
model.fit(train_x, train_y, nb_epoch=100, batch_size=1000, verbose=2, validation_data=(test_x, test_y))

为了完整起见，这里是损失函数的代码:

For completeness, here's the code for the loss function:

def weibull_loglik_discrete(y_true, ab_pred, name=None):
    y_ = y_true[:, 0]
    u_ = y_true[:, 1]
    a_ = ab_pred[:, 0]
    b_ = ab_pred[:, 1]

    hazard0 = k.pow((y_ + 1e-35) / a_, b_)
    hazard1 = k.pow((y_ + 1) / a_, b_)

    return -1 * k.mean(u_ * k.log(k.exp(hazard1 - hazard0) - 1.0) - hazard1)

这是自定义激活函数的代码:

And here's the code for the custom activation function:

def activate(ab):
    a = k.exp(ab[:, 0])
    b = k.softplus(ab[:, 1])

    a = k.reshape(a, (k.shape(a)[0], 1))
    b = k.reshape(b, (k.shape(b)[0], 1))

    return k.concatenate((a, b), axis=1)

当我拟合模型并进行一些测试预测时，测试集中的每个样本都得到完全相同的预测，这看起来很可疑.

When I fit the model and make some test predictions, every sample in the test set gets exactly the same prediction, which seems fishy.

如果我移除遮罩层，事情会变得更好，这让我觉得遮罩层有问题，但据我所知，我完全遵循了文档.

Things get better if I remove the masking layer, which makes me think there's something wrong with the masking layer, but as far as I can tell, I've followed the documentation exactly.

掩蔽层是否有错误指定的内容?我还缺什么吗?

Is there something mis-specified with the masking layer? Am I missing something else?

推荐答案

我无法在没有实际数据的情况下进行验证，但我对 RNN 有类似的体验.在我的情况下，规范化解决了这个问题.为您的模型添加规范化层.

I could not validate without actual data, but I had a similar experience with an RNN. In my case normalization solved the issue. Add a normalization layer to your model.

这篇关于具有不同时间步长的用于 RNN 的 Keras 掩码的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！