将Dropout添加到测试/推断阶段 [英] Adding Dropout to testing/inference phase

问题描述

我已经在Keras中为某些时间序列训练了以下模型:

I've trained the following model for some timeseries in Keras:

    input_layer = Input(batch_shape=(56, 3864))
    first_layer = Dense(24, input_dim=28, activation='relu',
                        activity_regularizer=None,
                        kernel_regularizer=None)(input_layer)
    first_layer = Dropout(0.3)(first_layer)
    second_layer = Dense(12, activation='relu')(first_layer)
    second_layer = Dropout(0.3)(second_layer)
    out = Dense(56)(second_layer)
    model_1 = Model(input_layer, out)

然后，我定义了一个新模型，其中训练有素的层为model_1，并向其中添加了具有不同速率drp的辍学层:

Then I defined a new model with the trained layers of model_1 and added dropout layers with a different rate, drp, to it:

    input_2 = Input(batch_shape=(56, 3864))
    first_dense_layer = model_1.layers[1](input_2)
    first_dropout_layer = model_1.layers[2](first_dense_layer)
    new_dropout = Dropout(drp)(first_dropout_layer)
    snd_dense_layer = model_1.layers[3](new_dropout)
    snd_dropout_layer = model_1.layers[4](snd_dense_layer)
    new_dropout_2 = Dropout(drp)(snd_dropout_layer)
    output = model_1.layers[5](new_dropout_2)
    model_2 = Model(input_2, output)

然后我将得到以下两个模型的预测结果:

Then I'm getting the prediction results of these two models as follow:

result_1 = model_1.predict(test_data, batch_size=56)
result_2 = model_2.predict(test_data, batch_size=56)

我期望得到完全不同的结果，因为第二个模型具有新的辍学层，并且这两个模型是不同的(IMO)，但事实并非如此.两者都产生相同的结果.为什么会这样呢?

I was expecting to get completely different results because the second model has new dropout layers and theses two models are different (IMO), but that's not the case. Both are generating the same result. Why is that happening?

推荐答案

正如我在评论中提到的，Dropout层在推理阶段(即测试模式)处于关闭状态，因此，当您使用model.predict()时，Dropout图层未激活.但是，如果您希望有一个在训练和推理阶段都使用Dropout的模型，则可以在调用它时传递training参数，

As I mentioned in the comments, the Dropout layer is turned off in inference phase (i.e. test mode), so when you use model.predict() the Dropout layers are not active. However, if you would like to have a model that uses Dropout both in training and inference phase, you can pass training argument when calling it, as suggested by François Chollet:

# ...
new_dropout = Dropout(drp)(first_dropout_layer, training=True)
# ...

或者，如果您已经训练了模型，现在想在推理模式下使用它并保持Dropout层(以及可能在训练/推理阶段具有不同行为的其他层，例如BatchNormalization)，您可以定义一个后端函数，该函数接受模型的输入以及Keras学习阶段:

Alternatively, If you have already trained your model and now want to use it in inference mode and keep the Dropout layers (and possibly other layers which have different behavior in training/inference phase such as BatchNormalization) active, you can define a backend function that takes the model's inputs as well as Keras learning phase:

from keras import backend as K

func = K.function(model.inputs + [K.learning_phase()], model.outputs)

# to use it pass 1 to set the learning phase to training mode
outputs = func([input_arrays] + [1.]) 

这篇关于将Dropout添加到测试/推断阶段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！