使用fit_generator和valuate_generator训练网络时，如何绘制AUC和ROC? [英] How can I plot AUC and ROC while using fit_generator and evaluate_generator to train my network?

问题描述

我正在使用生成器来训练和预测我的数据分类.这是 ImageDataGenerator 的示例

I am using generator to train and predict classification on my data. Here is an example of ImageDataGenerator

from keras.preprocessing.image import ImageDataGenerator

batch_size = 16

train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)


train_generator = train_datagen.flow_from_directory(
        'data/train',  # this is the target directory
        target_size=(150, 150),  
        batch_size=batch_size,
        class_mode='binary') 

validation_generator = test_datagen.flow_from_directory(
        'data/validation',
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='binary')


model.fit_generator(
        train_generator,
        steps_per_epoch=2000 // batch_size,
        epochs=50,
        validation_data=validation_generator,
        validation_steps=800 // batch_size)
model.save_weights('first_try.h5')  # always save your weights after training or during training

我的问题如何创建

My question how can I create AUC and ROC when I use fit_generator?

推荐答案

在这种情况下，我认为您最好的选择是将AUC定义为新指标.为此，您必须在tensorflow中定义指标(我假设您正在使用tensorflow后端).

I think your best bet in this case is to define AUC as a new metric. To do this, you have to define the metric in tensorflow (I am assuming you are using tensorflow backend).

我以前尝试过的一种方法(但是，我不记得我对结果的正确性进行了测试)是这样的:

One way which I have experimented with previously (however, I don't recall I tested it for correctness of results) is something like this:

def as_keras_metric(method):
    """
    This is taken from:
    https://stackoverflow.com/questions/45947351/how-to-use-tensorflow-metrics-in-keras/50527423#50527423
    """
    @functools.wraps(method)
    def wrapper(*args, **kwargs):
        """ Wrapper for turning tensorflow metrics into keras metrics """
        value, update_op = method(*args, **kwargs)
        tf.keras.backend.get_session().run(tf.local_variables_initializer())
        with tf.control_dependencies([update_op]):
            value = tf.identity(value)
        return value
    return wrapper

，然后在编译模型时定义指标:

and then define the metric when the model is compiled:

model.compile(metrics=['accuracy', as_keras_metric(tf.metrics.auc)], optimizer='adam', loss='categorical_crossentropy')

尽管这会吐出一些数字，但我还没有找出它们是否正确.如果您能够对此进行测试，并且能够给出正确的结果，请告诉我，我很想找出答案.

Although this spits out numbers, I have yet to find out if they are correct. If you are able to test this, and it gives the correct results, or not, please let me know, I would be interested to find out.

解决此问题的第二种方法是使用回调类，并至少定义on_epoch_end函数，然后可以从那里调用sklearn roc_auc_score并打印或保存到日志.

A second way to go around this is to use a callback class and define at least the on_epoch_end function, and then then you can call sklearn roc_auc_score from there and either print out or save to a log.

但是，到目前为止，我发现您需要通过__init__向其提供训练数据，因此对于生成器，您需要确保回调的生成器提供的数据与模型拟合的数据相同发电机.另一方面，对于验证生成器，可以使用self.validation_data从回调类访问它，该回调类与提供给fit_generator的相同.

However, what I have found out so far, is that you need to provide it the training data through __init__, and thus with generators, you need to make sure the callback's generator is supplying the same data as the model's fitting generator. On the other hand, for the validation generator, it can be accessed from a callback class using self.validation_data, which is the same as the one supplied to fit_generator.

这篇关于使用fit_generator和valuate_generator训练网络时，如何绘制AUC和ROC?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！