具有多个输入的Tensorflow 2.0自定义损失功能 [英] Tensorflow 2.0 Custom loss function with multiple inputs

问题描述

我正在尝试使用以下两个损失函数来优化模型

I am trying to optimize a model with the following two loss functions

def loss_1(pred, weights, logits):
    weighted_sparse_ce = kls.SparseCategoricalCrossentropy(from_logits=True)
    policy_loss = weighted_sparse_ce(pred, logits, sample_weight=advantages)

和

def loss_2(y_pred, y):
    return kls.mean_squared_error(y_pred, y)

但是，因为TensorFlow 2期望损失函数的形式为

however, because TensorFlow 2 expects loss function to be of the form

def fn(y_pred, y_true):
    ...

我正在使用loss_1的变通办法，其中将pred和weights打包到单个张量中，然后在调用loss_1之前.org/versions/r2.0/api_docs/python/tf/keras/Model#fit"rel =" nofollow noreferrer> model.fit ，然后将其解压缩为loss_1.这是不雅和令人讨厌的，因为pred和weights是不同的数据类型，因此每次我呼叫

I am using a work-around for loss_1 where I pack pred and weights into a single tensor before passing to loss_1 in the call to model.fit and then unpack them in loss_1. This is inelegant and nasty because pred and weights are of different data types and so this requires an additional cast, pack, un-pack and un-cast each time I call model.fit.

此外，我知道fit的sample_weight参数，这有点像

Furthermore, I am aware of the sample_weight argument to fit, which is kind of like the solution to this question. This might be a workable solution were it not for the fact that I am using two loss functions and I only want the sample_weight applied to one of them. Also, even if this were a solution, would it not be generalizable to other types of custom loss functions.

简而言之，我的问题是:

All that being said, my question, said concisely, is:

创建任意数量的损失函数的最佳方法是什么 TensorFlow 2中的参数?

What is the best way to create a loss function with an arbitrary number of arguments in TensorFlow 2?

我尝试过的另一件事是传递 tf.tuple ，但这似乎也违反了TensorFlow的损失函数输入要求.

Another thing I have tried is passing a tf.tuple but that also seems to violate TensorFlow's desires for a loss function input.

推荐答案

可以使用自定义培训.您只需要在GradientTape上下文中计算两分量损失函数，然后使用产生的梯度调用优化器即可.例如，您可以创建一个函数custom_loss，计算给定每个参数的两个损耗:

This problem can be easily solved using custom training in TF2. You need only compute your two-component loss function within a GradientTape context and then call an optimizer with the produced gradients. For example, you could create a function custom_loss which computes both losses given the arguments to each:

def custom_loss(model, loss1_args, loss2_args):
  # model: tf.model.Keras
  # loss1_args: arguments to loss_1, as tuple.
  # loss2_args: arguments to loss_2, as tuple.
  with tf.GradientTape() as tape:
    l1_value = loss_1(*loss1_args)
    l2_value = loss_2(*loss2_args)
    loss_value = [l1_value, l2_value]
  return loss_value, tape.gradient(loss_value, model.trainable_variables)

# In training loop:
loss_values, grads = custom_loss(model, loss1_args, loss2_args)
optimizer.apply_gradients(zip(grads, model.trainable_variables))

通过这种方式，每个损失函数都可以采用任意数量的渴望张量，无论它们是模型的输入还是输出.每个损失函数的参数集不必像本示例中所示那样是不相交的.

In this way, each loss function can take an arbitrary number of eager tensors, regardless of whether they are inputs or outputs to the model. The sets of arguments to each loss function need not be disjoint as shown in this example.

这篇关于具有多个输入的Tensorflow 2.0自定义损失功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！