每10个周期报告一次Keras模型评估指标吗? [英] Report Keras model evaluation metrics every 10 epochs?

问题描述

我想知道我模型的特异性和敏感性.目前，我正在评估所有时期之后的模型:

I'd like to know the specificity and sensitivity of my model. Currently, I'm evaluating the model after all epochs are finished:

from sklearn.metrics import confusion_matrix

predictions = model.predict(x_test)
y_test = np.argmax(y_test, axis=-1)
predictions = np.argmax(predictions, axis=-1)
c = confusion_matrix(y_test, predictions)
print('Confusion matrix:\n', c)
print('sensitivity', c[0, 0] / (c[0, 1] + c[0, 0]))
print('specificity', c[1, 1] / (c[1, 1] + c[1, 0]))

这种方法的缺点是，只有在培训结束后我才能得到我关心的输出.宁愿每10个周期左右获取一次指标.

The disadvantage of this approach, is I only get the output I care about when training has finished. Would prefer to get metrics every 10 epochs or so.

BTW:尝试过 metrics=[]此处.可能要走回调?

BTW: Tried with the metrics=[] here. Possibly a callback is the way to go?

推荐答案

自定义回调是一个很好的解决方案，可以让您充分控制培训过程.类似于以下内容:

A custom Callback would be a nice solution giving you enough control over the training procedure. Something along the lines of:

class SensitivitySpecificityCallback(Callback):
    def on_epoch_end(self, epoch, logs=None):
        if epoch % 10 == 1:
            x_test = self.validation_data[0]
            y_test = self.validation_data[1]
            # x_test, y_test = self.validation_data
            predictions = self.model.predict(x_test)
            y_test = np.argmax(y_test, axis=-1)
            predictions = np.argmax(predictions, axis=-1)
            c = confusion_matrix(y_test, predictions)

            print('Confusion matrix:\n', c)
            print('sensitivity', c[0, 0] / (c[0, 1] + c[0, 0]))
            print('specificity', c[1, 1] / (c[1, 1] + c[1, 0]))

其中epoch是纪元数，logs包含通常的指标+模型训练的损失.

where epoch is the epoch number and logs contain the usual metrics + loss the model trains.

然后运行:

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          shuffle='batch',
          validation_data=(x_test, y_test),
          callbacks=[SensitivitySpecificityCallback()])

注意:如果您不喜欢根据指标对模型进行训练，则可以使用以下方法缩短训练时间:

NOTE: if you don't like how your model is training based on your metrics you can cut the training short with:

self.model.stop_training = True

这将为您停止培训.

这篇关于每10个周期报告一次Keras模型评估指标吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！