如何在Keras模型中停用通过training = True调用的辍学层? [英] How to deactivate a dropout layer called with training=True in a Keras model?

问题描述

我希望查看训练tf.keras模型的最终输出.在这种情况下，它将是来自softmax函数的一系列预测，例如[0,0,0,1,0,1].

I wish to view the final output of training a tf.keras model. In this case it would be an array of predictions from the softmax function, e.g. [0,0,0,1,0,1].

此处的其他线程建议使用model.predict(training_data)，但这不适用于我的情况，因为我在训练和验证时使用了辍学功能，因此神经元被随机丢弃，并使用相同的数据再次进行预测结果不同.

Other threads on here have suggested using model.predict(training_data), but this won't work for my situation since I am using dropout at training and validation, so neurons are randomly dropped and predicting again with the same data will give a different result.

def get_model():
    inputs = tf.keras.layers.Input(shape=(input_dims,))
    x = tf.keras.layers.Dropout(rate=dropout_rate)(inputs, training=True)
    x = tf.keras.layers.Dense(units=29, activation='relu')(x)
    x = tf.keras.layers.Dropout(rate=dropout_rate)(x, training=True)  
    x = tf.keras.layers.Dense(units=15, activation='relu')(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)
    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',      
                  metrics=['sparse_categorical_accuracy'])
    return model

myModel = get_model()
myModel.summary()
myModel.fit(X_train, y_train,
           batch_size = batch_size,
           epochs= epochs,
           verbose = 1,
           validation_data = (X_val, y_val))

在tensorflow中，您可以在轻松训练后获取模型的输出.这是来自

In tensorflow, you can grab the output of a model after training quite easily. Here is an example from a Github repo:

input = tf.placeholder(tf.float32, shape=[None, INPUT_DIMS])
labels = tf.placeholder(tf.float32, shape=[None])

hidden = tf.nn.tanh(make_nn_layer(normalized, NUM_HIDDEN))
logits = make_nn_layer(hidden, NUM_CLASSES)
outputs = tf.argmax(logits, 1)

int_labels = tf.to_int64(labels)
cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits, int_labels, name='xentropy')
train_step = tf.train.AdamOptimizer().minimize(cross_entropy)

correct_prediction = tf.equal(outputs, int_labels)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

with tf.Session() as sess:
    sess.run(tf.initialize_all_variables())

    validation_dict = {
        input: validation_data[:,0:7],
        labels: validation_data[:,7],}

    for i in range(NUM_BATCHES):
        batch = training_data[numpy.random.choice(training_size, BATCH_SIZE, False),:]
        train_step.run({input: batch[:,0:7], labels: batch[:,7]})

        if i % 100 == 0 or i == NUM_BATCHES - 1:
            print('Accuracy %.2f%% at step %d' % (accuracy.eval(validation_dict) * 100, i))

    output_data = outputs.eval({input: data_vector[:,0:7]})

我只能从经过训练的模型中得到的输出似乎是一个历史对象.还有一个myModel.output对象，但这是一个张量，如果不将数据放入其中就无法评估.有什么想法吗?

The only output I can get from the trained model appears to be a history object. There is also a myModel.output object, but it is a tensor that I can't evaluate without putting data into it. Any ideas?

推荐答案

据我所知，调用图层时传递training=True后，您无法关闭Dropout(除非您将权重转移到新模型中)具有相同的架构).但是，相反，您可以在正常情况下(即在调用中不使用training参数)构建和训练模型，然后在测试阶段通过定义一个后端函数(即keras.backend.function())有选择地打开和关闭退出层.设置学习阶段(即keras.backend.learning_phase()):

As far as I know, you can't turn off the dropout after passing training=True when calling the layers (unless you transfer the weights to a new model with the same architecture). However, instead you can build and train your model in normal case (i.e. without using training argument in the calls) and then selectively turn on and off the dropout layer in test phase by defining a backend function (i.e. keras.backend.function()) and setting the learning phase (i.e. keras.backend.learning_phase()):

# build your model normally (i.e. without using `training=True` argument)

# train your model...

from keras import backend as K

func = K.function(model.inputs + [K.learning_phase()], model.outputs)

# run the model with dropout layers being active, i.e. learning_phase == 1
preds = func(list_of_input_arrays + [1])

# run the model with dropout layers being inactive, i.e. learning_phase == 0
preds = func(list_of_input_arrays + [0])

更新:如上所述，另一种方法是定义具有相同体系结构但不设置training=True的新模型，然后将权重从训练有素的模型转移到该新模型.为此，我只需在您的get_model()函数中添加一个training参数:

Update: As I suggested above, another approach is to define a new model with the same architecture but without setting training=True, and then transfer the weights from the trained model to this new model. To achieve this, I just add a training argument to your get_model() function:

def get_model(training=None):
    inputs = tf.keras.layers.Input(shape=(input_dims,))
    x = tf.keras.layers.Dropout(rate=dropout_rate)(inputs, training=training)
    x = tf.keras.layers.Dense(units=29, activation='relu')(x)
    x = tf.keras.layers.Dropout(rate=dropout_rate)(x, training=training)  
    x = tf.keras.layers.Dense(units=15, activation='relu')(x)
    outputs = tf.keras.layers.Dense(2, activation='softmax')(x)
    model = tf.keras.Model(inputs=inputs, outputs=outputs)
    model.compile(optimizer='adam',
                  loss='sparse_categorical_crossentropy',      
                  metrics=['sparse_categorical_accuracy'])
    return model

# build a model with dropout layers active in both training and test phases
myModel = get_model(training=True)
# train the model
myModel.fit(...)

# build a clone of the model with dropouts deactivated in test phase
myTestModel = get_model()  # note: the `training` is `None` by default
# transfer the weights from the trained model to this model
myTestModel.set_weights(myModel.get_weights())
# use the new model in test phase; the dropouts would not be active
myTestModel.predict(...)

这篇关于如何在Keras模型中停用通过training = True调用的辍学层?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！