具有不同时间步长的用于 RNN 的 Keras 掩码 [英] Keras Masking for RNN with Varying Time Steps
问题描述
我正在尝试使用具有不同时间长度的序列在 Keras 中拟合 RNN.我的数据位于格式 (sample, time, feature) = (20631, max_time, 24)
的 Numpy 数组中,其中 max_time
在运行时确定为具有最多时间戳的样本可用的时间步长.我已经用 0
填充了每个时间序列的开头,显然最长的除外.
I'm trying to fit an RNN in Keras using sequences that have varying time lengths. My data is in a Numpy array with format (sample, time, feature) = (20631, max_time, 24)
where max_time
is determined at run-time as the number of time steps available for the sample with the most time stamps. I've padded the beginning of each time series with 0
, except for the longest one, obviously.
我最初是这样定义我的模型的...
I've initially defined my model like so...
model = Sequential()
model.add(Masking(mask_value=0., input_shape=(max_time, 24)))
model.add(LSTM(100, input_dim=24))
model.add(Dense(2))
model.add(Activation(activate))
model.compile(loss=weibull_loglik_discrete, optimizer=RMSprop(lr=.01))
model.fit(train_x, train_y, nb_epoch=100, batch_size=1000, verbose=2, validation_data=(test_x, test_y))
为了完整起见,这里是损失函数的代码:
For completeness, here's the code for the loss function:
def weibull_loglik_discrete(y_true, ab_pred, name=None):
y_ = y_true[:, 0]
u_ = y_true[:, 1]
a_ = ab_pred[:, 0]
b_ = ab_pred[:, 1]
hazard0 = k.pow((y_ + 1e-35) / a_, b_)
hazard1 = k.pow((y_ + 1) / a_, b_)
return -1 * k.mean(u_ * k.log(k.exp(hazard1 - hazard0) - 1.0) - hazard1)
这是自定义激活函数的代码:
And here's the code for the custom activation function:
def activate(ab):
a = k.exp(ab[:, 0])
b = k.softplus(ab[:, 1])
a = k.reshape(a, (k.shape(a)[0], 1))
b = k.reshape(b, (k.shape(b)[0], 1))
return k.concatenate((a, b), axis=1)
当我拟合模型并进行一些测试预测时,测试集中的每个样本都得到完全相同的预测,这看起来很可疑.
When I fit the model and make some test predictions, every sample in the test set gets exactly the same prediction, which seems fishy.
如果我移除遮罩层,事情会变得更好,这让我觉得遮罩层有问题,但据我所知,我完全遵循了文档.
Things get better if I remove the masking layer, which makes me think there's something wrong with the masking layer, but as far as I can tell, I've followed the documentation exactly.
掩蔽层是否有错误指定的内容?我还缺什么吗?
Is there something mis-specified with the masking layer? Am I missing something else?
推荐答案
我无法在没有实际数据的情况下进行验证,但我对 RNN 有类似的体验.在我的情况下,规范化解决了这个问题.为您的模型添加规范化层.
I could not validate without actual data, but I had a similar experience with an RNN. In my case normalization solved the issue. Add a normalization layer to your model.
这篇关于具有不同时间步长的用于 RNN 的 Keras 掩码的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!