Keras-Nan总结直方图LSTM [英] Keras - Nan in summary histogram LSTM
问题描述
我已经使用Keras和LeakyReLU高级激活编写了一个LSTM模型:
I've written an LSTM model using Keras, and using LeakyReLU advance activation:
# ADAM Optimizer with learning rate decay
opt = optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0001)
# build the model
model = Sequential()
num_features = data.shape[2]
num_samples = data.shape[1]
model.add(
LSTM(16, batch_input_shape=(None, num_samples, num_features), return_sequences=True, activation='linear'))
model.add(LeakyReLU(alpha=.001))
model.add(Dropout(0.1))
model.add(LSTM(8, return_sequences=True, activation='linear'))
model.add(Dropout(0.1))
model.add(LeakyReLU(alpha=.001))
model.add(Flatten())
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer=opt,
metrics=['accuracy', keras_metrics.precision(), keras_metrics.recall(), f1])
我的数据是一个平衡的二进制标签集.即:50％标记为1 50％标记为0.在LeakyReLU激活之前，我已将activation='linear'用于LSTM层，类似于
My data is a balanced binary labeled set. i.e: 50% labeled 1 50% labeled 0. I've used activation='linear' for the LSTM layers preceding the LeakyReLU activation, similar to this example I found on GitHub.
该模型在该配置中引发Nan in summary histogram错误.将LSTM激活更改为activation='sigmoid'效果很好，但似乎做错了事.
The model throws Nan in summary histogram error in that configuration. Changing the LSTM activations to activation='sigmoid' works well, but seems like the wrong thing to do.
阅读此StackOverflow 问题建议"在计算损失时引入一个小值"，我不确定如何在内置的损失函数中做到这一点.
Reading this StackOverflow question suggested "introducing a small value when computing the loss", I'm just not sure how to do it on a built-in loss function.
任何帮助/解释将不胜感激.
Any help/explanation would be appreciated.
更新: 我可以看到损失在第一个时期是微不足道的
Update: I can see that the loss is nan on the first epoch
260/260 [==============================] - 6s 23ms/step -
loss: nan - acc: 0.5000 - precision: 0.5217 - recall: 0.6512 - f1: nan - val_loss: nan - val_acc: 0.0000e+00 - val_precision: -2147483648.0000 - val_recall: -49941480.1860 - val_f1: nan
更新2 我已经同时升级了TensorFlow和Keras版本1.12.0及更高版本2.2.4.没有效果.
Update 2 I've upgraded both TensorFlow & Keras to versions 1.12.0 & 2.2.4 . There was no effect.
我还尝试按照@Oluwafemi Sule的建议在第一LSTM层上增加损耗，这看起来像是朝着正确方向迈出的一步，现在，损耗在第一个时期已经不成立了，但是，我仍然遇到相同的错误...可能是因为其他nan值，例如val_loss/val_f1.
I also tried adding a loss to the first LSTM layer as suggested by @Oluwafemi Sule, it looks like a step in the right direction, now the loss is not nan on the first epoch, however, I still get the same error ... probably because of other nan values, like the val_loss / val_f1.
[==============================] - 7s 26ms/step -
loss: 1.9099 - acc: 0.5077 - precision: 0.5235 - recall: 0.6544 - f1: 0.5817 - val_loss: nan - val_acc: 0.5172 - val_precision: 35.0000 - val_recall: 0.9722 - val_f1: nan
更新3 我尝试仅使用精度指标来编译网络，但没有成功:
Update 3 I tried to compile the network with just the accuracy metric, with no success:
Epoch 1/300
260/260 [==============================] - 8s 29ms/step - loss: nan - acc: 0.5538 - val_loss: nan - val_acc: 0.0000e+00
推荐答案
此答案从建议开始，即在计算损失时引入一个较小的值.
This answers starts from the suggestion to introduce a small value when computing the loss.
keras.layers.LSTM as with all layers that are direct or indirect subclasses of keras.engine.base_layer.Layer has a add_loss method that can be used to set a starting value for the loss.
我建议对LSTM层执行此操作，看看它对您的结果是否有任何影响.
I suggest to do this for the LSTM layer and see if it makes any difference for your results.
lstm_layer = LSTM(8, return_sequences=True, activation='linear')
lstm_layer.add_loss(1.0)
model.add(lstm_layer)
这篇关于Keras-Nan总结直方图LSTM的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
