利用非训练数据的训练权值设计新的损失函数 [英] Using training weights on a non-training data to design a new loss function

问题描述

我希望在训练迭代中访问训练点，并通过使用训练集中未包括的数据点将软约束合并到我的损失函数中。我将使用this post作为参考。

import numpy as np
import keras.backend as K
from keras.layers import Dense, Input
from keras.models import Model

# Some random training data and labels
features = np.random.rand(100, 5)
labels = np.random.rand(100, 2)

# Simple neural net with three outputs
input_layer = Input((20,))
hidden_layer = Dense(16)(input_layer)
output_layer = Dense(3)(hidden_layer)


# Model
model = Model(inputs=input_layer, outputs=output_layer)


#each training point has another data pair. In the real example, I will have multiple 
#supporters. That is why I am using dict.

holder =  np.random.rand(100, 5)
iter = np.arange(start=1, stop=features.shape[0], step=1)
supporters = {}

for i,j in zip(iter, holder): #i represent the ith training data
    supporters[i]=j


# Write a custom loss function
def custom_loss(y_true, y_pred):
    # Normal MSE loss
    mse = K.mean(K.square(y_true-y_pred), axis=-1)
    new_constraint = .... 

       

    return(mse+new_constraint)


model.compile(loss=custom_loss, optimizer='sgd')
model.fit(features, labels, epochs=1, ,batch_size=1=1)

为简单起见，让我们假设我希望通过使用固定的网络权重来最小化supporters中存储的配对数据的预测值和预测值之间的最小绝对值差。另外，假设我每批通过一个训练点。然而，我想不出该怎么做这个手术。我尝试了下面显示的内容，但很明显，它不正确。

new_constraint = K.sum(y_pred - model.fit(supporters))

推荐答案

Fit是培训评估模型的过程。我认为对于您的问题，最好使用当前权重加载模型的新实例并评估批量损失，以便计算主模型的损失。

main_model = Model()  # This is your main training model 

def custom_loss_1(y_true, y_pred):  # Avoid recursive calls
    mse = K.mean(K.square(y_true-y_pred), axis=-1)
    return mse

def custom_loss(y_true, y_pred):
    support_model =  tf.keras.models.clone_model(main_model)  # You copy the main model but the weights are uninitialized
    support_model.build((20,)) # You build with inputs same as your support data
    support_model.compile(loss=custom_loss_1, optimizer='sgd') 
    support_model.set_weights(main_model.get_weights())  # You  load the weight of the main model

    mse = custom_loss_1(y_true, y_pred)
    # You just want to evaluate the model, not to train. If you have more
    # metrics than just loss the use support_model.evaluate(supporters)[0]
    new_constraint = K.sum(y_pred -  support_model.predict(supporters))  # predict to get the output, evaluate to get the metrics

    return(mse+new_constraint)

这篇关于利用非训练数据的训练权值设计新的损失函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！