在Keras中使用ImageDataGenerator时，如何定义依赖于输入的自定义成本函数? [英] How to define custom cost function that depends on input when using ImageDataGenerator in Keras?

问题描述

我想定义一个自定义成本函数

I would like to define a custom cost function

def custom_objective(y_true, y_pred):
    ....
    return L

不仅取决于y_true和y_pred，而且取决于产生y_pred的相应x的某些功能.我能想到的唯一方法是隐藏" y_true中的相关功能，以便y_true = [usual_y_true, relevant_x_features]或类似的东西.

that will depend not only on y_true and y_pred, but on some feature of the corresponding x that produced y_pred. The only way I can think of doing this is to "hide" the relevant features in y_true, so that y_true = [usual_y_true, relevant_x_features], or something like that.

实施此操作存在两个主要问题:

There are two main problems I am having with implementing this:

1)更改y_true的形状意味着我需要在y_pred上填充一些垃圾，以使它们的形状相同.我可以通过修改模型的最后一层来做到这一点

1) Changing the shape of y_true means I need to pad y_pred with some garbage so that their shapes are the same. I can do this by modyfing the last layer of my model

2)我像这样使用数据增强:

2) I used data augmentation like so:

datagen = ImageDataGenerator(preprocessing_function=my_augmenter)

其中，my_augmenter()是该函数，还应该为我提供上述custom_objective()中要使用的相关x功能.但是，

where my_augmenter() is the function that should also give me the relevant x features to use in custom_objective() above. However, training with

model.fit_generator(datagen.flow(x_train, y_train, batch_size=1), ...)

似乎没有让我使用my_augmenter计算出的功能.

doesn't seem to give me access to the features calculated with my_augmenter.

我想我可以隐藏增强的x_train中的功能，立即在模型设置中将其复制，然后将其直接输入到y_true或类似的内容中，但是肯定有更好的方法这个吗?

I suppose I could hide the features in the augmented x_train, copy them right away in my model setup, and then feed them directly into y_true or something like that, but surely there must be a better way to do this?

推荐答案

也许您可以使用以下方法创建一个两部分模型:

Maybe you could create a two part model with:

• 内部模型:可以预测所需输出的原始模型
• 外部型号:
  • 将y_true数据作为输入
  • 将要素作为输入
  • 输出损失本身(而不是预测数据)
  • Inner model: original model that predicts desired outputs
  • Outer model:
    • Takes y_true data as inputs
    • Takes features as inputs
    • Outputs the loss itself (instead of predicted data)

    因此，假设您已经定义了originalModel.让我们定义外部模型.

    So, suppose you already have the originalModel defined. Let's define the outer model.

    #this model has three inputs:
    originalInputs = originalModel.input  
    yTrueInputs = Input(shape_of_y_train)
    featureInputs = Input(shape_of_features)
    
    #the original outputs will become an input for a custom loss layer
    originalOutputs = originalModel.output
    
    #this layer contains our custom loss
    loss = Lambda(innerLoss)([originalOutputs, yTrueInputs, featureInputs])
    
    #outer model
    outerModel = Model([originalInputs, yTrueInputs, featureInputs], loss)
    

    现在，我们自定义的内部损失:

    Now, our custom inner loss:

    def innerLoss(x):
        y_pred = x[0] 
        y_true = x[1]
        features = x[2] 
    
        .... calculate and return loss here .... 
    

    现在，对于已经在其中包含"自定义损失的模型，我们实际上并不需要最终损失函数，但是由于keras要求它，我们将最终损失仅用作return y_pred:

    Now, for this model that already contains a custom loss "inside" it, we don't actually want a final loss function, but since keras demands it, we will use the final loss as just return y_pred:

    def finalLoss(true,pred):
        return pred
    

    这将允许我们训练仅通过虚拟y_true的情况.

    This will allow us to train passing just a dummy y_true.

    但是，当然，我们还需要一个自定义生成器，否则我们将无法获得功能.

    But of course, we also need a custom generator, otherwise we can't get the features.

    考虑您已经定义了originalGenerator =datagen.flow(x_train, y_train, batch_size=1):

    def customGenerator(originalGenerator):
    
        while True: #keras needs infinite generators
            x, y = next(originalGenerator)
    
            features = ____extract features here____(x)
    
            yield (x,y,features), y 
                #the last y will be a dummy output, necessary but not used
    

    如果还需要额外的随机批处理功能并使用多处理功能，则可以按照相同的逻辑实现class CustomGenerator(keras.utils.Sequence). 帮助页面显示了操作方法.

    You could also, if you want the extra functionality of randomizing batch order and use multiprocessing, implement a class CustomGenerator(keras.utils.Sequence) following the same logic. The help page shows how.

    因此，让我们编译并训练外部模型(这也会训练内部模型，以便您以后可以使用它进行预测):

    So, let's compile and train the outer model (this also trains the inner model so you can use it later for predicting):

    outerModel.compile(optimizer=..., loss=finalLoss)
    outerModel.fit_generator(customGenerator(originalGenerator), batchesInOriginalGenerator, 
                             epochs=...)
    

    这篇关于在Keras中使用ImageDataGenerator时，如何定义依赖于输入的自定义成本函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！