每个张量组的Keras自定义损失函数 [英] Keras custom loss function per tensor group

问题描述

我正在编写一个自定义损失函数，该函数需要计算每组预测值的比率.作为简化的示例，这是我的数据和模型代码的样子:

I am writing a custom loss function that requires calculating ratios of predicted values per group. As a simplified example, here is what my Data and model code looks like:

def main():
    df = pd.DataFrame(columns=["feature_1", "feature_2", "condition_1", "condition_2", "label"],
                      data=[[5, 10, "a", "1", 0],
                            [30, 20, "a", "1", 1],
                            [50, 40, "a", "1", 0],
                            [15, 20, "a", "2", 0],
                            [25, 30, "b", "2", 1],
                            [35, 40, "b", "1", 0],
                            [10, 80, "b", "1", 1]])
    features = ["feature_1", "feature_2"]
    conds_and_label = ["condition_1", "condition_2", "label"]
    X = df[features]
    Y = df[conds_and_label]
    model = my_model(input_shape=len(features))
    model.fit(X, Y, epochs=10, batch_size=128)
    model.evaluate(X, Y)


def custom_loss(conditions, y_pred):  # this is what I need help with
    conds = ["condition_1", "condition_2"]
    conditions["label_pred"] = y_pred
    g = conditions.groupby(by=conds,
                           as_index=False).apply(lambda x: x["label_pred"].sum() /
                                                           len(x)).reset_index(name="pred_ratio")
    # true_ratios will be a constant, external DataFrame. Simplified example here:
    true_ratios = pd.DataFrame(columns=["condition_1", "condition_2", "true_ratio"],
                               data=[["a", "1", 0.1],
                                     ["a", "2", 0.2],
                                     ["b", "1", 0.8],
                                     ["b", "2", 0.9]])
    merged = pd.merge(g, true_ratios, on=conds)
    merged["diff"] = merged["pred_ratio"] - merged["true_ratio"]
    return K.mean(K.abs(merged["diff"]))


def joint_loss(conds_and_label, y_pred):
    y_true = conds_and_label[:, 2]
    conditions = tf.gather(conds_and_label, [0, 1], axis=1)
    loss_1 = standard_loss(y_true=y_true, y_pred=y_pred)  # not shown
    loss_2 = custom_loss(conditions=conditions, y_pred=y_pred)
    return 0.5 * loss_1 + 0.5 * loss_2


def my_model(input_shape=None):
    model = Sequential()
    model.add(Dense(units=2, activation="relu"), input_shape=(input_shape,))
    model.add(Dense(units=1, activation='sigmoid'))
    model.add(Flatten())
    model.compile(loss=joint_loss, optimizer="Adam",
                  metrics=[joint_loss, custom_loss, "accuracy"])
    return model

我需要帮助的是custom_loss函数.如您所见，它当前的编写方式就像输入是Pandas DataFrames.但是，输入将是Keras张量(具有tensorflow后端)，因此我试图弄清楚如何在custom_loss中转换当前代码以使用Keras/TF后端函数.例如，我在网上搜索，却找不到在Keras/TF中进行分组的方法，以获得我需要的比率...

What I need help with is the custom_loss function. As you can see, it is currently written as if the inputs are Pandas DataFrames. However, the inputs will be Keras Tensors (with tensorflow backend), so I am trying to figure out how to convert the current code in custom_loss to use Keras/TF backend functions. For example, I searched online and couldn't find out a way to do a groupby in Keras/TF to get the ratios I need...

一些可能对您有所帮助的上下文/解释:

Some context/explanation that might be helpful to you:

1. 我的主要损失函数是joint_loss，它由standard_loss(未显示)和custom_loss组成.但是我只需要转换custom_loss的帮助.
2. custom_loss的作用是:

1. My main loss function is joint_loss, which consists of standard_loss (not shown) and custom_loss. But I only need help converting custom_loss.
2. What custom_loss does is:

1. 在两个条件列上使用Groupby(这两个列代表数据组).
2. 获取每组中预测的1s与批次样品总数的比率.
3. 比较"pred_ratio"；到一组"true_ratio"；并获得差异.
4. 根据差异计算平均绝对误差.

推荐答案

我最终想出了一个解决方案，尽管我希望对此有一些反馈(特别是某些部分).解决方法如下:

I ended up figuring out a solution to this, though I would like some feedback on it (specifically some parts). Here is the solution:

import pandas as pd
import tensorflow as tf
import keras.backend as K
from keras.models import Sequential
from keras.layers import Dense, Flatten, Dropout
from tensorflow.python.ops import gen_array_ops


def main():
    df = pd.DataFrame(columns=["feature_1", "feature_2", "condition_1", "condition_2", "label"],
                      data=[[5, 10, "a", "1", 0],
                            [30, 20, "a", "1", 1],
                            [50, 40, "a", "1", 0],
                            [15, 20, "a", "2", 0],
                            [25, 30, "b", "2", 1],
                            [35, 40, "b", "1", 0],
                            [10, 80, "b", "1", 1]])
    df = pd.concat([df] * 500)  # making data artificially larger
    true_ratios = pd.DataFrame(columns=["condition_1", "condition_2", "true_ratio"],
                               data=[["a", "1", 0.1],
                                     ["a", "2", 0.2],
                                     ["b", "1", 0.8],
                                     ["b", "2", 0.9]])
    features = ["feature_1", "feature_2"]
    conditions = ["condition_1", "condition_2"]
    conds_ratios_label = conditions + ["true_ratio", "label"]
    df = pd.merge(df, true_ratios, on=conditions, how="left")
    X = df[features]
    Y = df[conds_ratios_label]
    # need to convert strings to ints because tensors can't mix strings with floats/ints
    mapping_1 = {"a": 1, "b": 2}
    mapping_2 = {"1": 1, "2": 2}
    Y.replace({"condition_1": mapping_1}, inplace=True)
    Y.replace({"condition_2": mapping_2}, inplace=True)
    X = tf.convert_to_tensor(X)
    Y = tf.convert_to_tensor(Y)
    model = my_model(input_shape=len(features))
    model.fit(X, Y, epochs=1, batch_size=64)
    print()
    print(model.evaluate(X, Y))


def custom_loss(conditions, true_ratios, y_pred):
    y_pred = tf.sigmoid((y_pred - 0.5) * 1000)
    uniques, idx, count = gen_array_ops.unique_with_counts_v2(conditions, [0])
    num_unique = tf.size(count)
    sums = tf.math.unsorted_segment_sum(data=y_pred, segment_ids=idx, num_segments=num_unique)
    lengths = tf.cast(count, tf.float32)
    pred_ratios = tf.divide(sums, lengths)
    mean_pred_ratios = tf.math.reduce_mean(pred_ratios)
    mean_true_ratios = tf.math.reduce_mean(true_ratios)
    diff = mean_pred_ratios - mean_true_ratios
    return K.mean(K.abs(diff))


def standard_loss(y_true, y_pred):
    return tf.losses.binary_crossentropy(y_true=y_true, y_pred=y_pred)


def joint_loss(conds_ratios_label, y_pred):
    y_true = conds_ratios_label[:, 3]
    true_ratios = conds_ratios_label[:, 2]
    conditions = tf.gather(conds_ratios_label, [0, 1], axis=1)
    loss_1 = standard_loss(y_true=y_true, y_pred=y_pred)
    loss_2 = custom_loss(conditions=conditions, true_ratios=true_ratios, y_pred=y_pred)
    return 0.5 * loss_1 + 0.5 * loss_2


def my_model(input_shape=None):
    model = Sequential()
    model.add(Dropout(0, input_shape=(input_shape,)))
    model.add(Dense(units=2, activation="relu"))
    model.add(Dense(units=1, activation='sigmoid'))
    model.add(Flatten())
    model.compile(loss=joint_loss, optimizer="Adam",
                  metrics=[joint_loss, "accuracy"],  # had to remove custom_loss because it takes 3 args now
                  run_eagerly=True)
    return model


if __name__ == '__main__':
    main()

主要更新是对custom_loss.我从custom_loss中删除了创建true_ratios数据框架，而是将其附加到main中的Y中.现在custom_loss需要3个参数，其中之一是true_ratios张量.我必须使用gen_array_ops.unique_with_counts_v2和unsorted_segment_sum来获取每组条件的总和.然后我得到每个组的长度以创建pred_ratios(基于y_pred的每个组的计算比率).最后，我得到了平均预测比率和平均真实比率，并取绝对差得到我的客户损失.

The main updates are to custom_loss. I removed creating the true_ratios DataFrame from custom_loss and instead appended it to my Y in main. Now custom_loss takes 3 arguments, one of which is the true_ratios tensor. I had to use gen_array_ops.unique_with_counts_v2 and unsorted_segment_sum to get sums per group of conditions. And then I got the lengths of each group in order to create pred_ratios (calculated ratios per group based on y_pred). Finally I get the mean predicted ratios and mean true ratios, and take the absolute difference to get my custom loss.

一些注意事项:

1. 由于模型的最后一层是S型，因此我的y_pred值介于0和1之间.因此，为了计算自定义损失中需要的比率，我需要将它们转换为0和1.最初，我尝试使用tf.round，但是我意识到这是不可区分的.因此，我改为将其替换为custom_loss内部的y_pred = tf.sigmoid((y_pred - 0.5) * 1000).本质上，这会将所有y_pred值设为0和1，但采用可微分的方式.似乎有点"hack".但是，如果对此有任何反馈，请告诉我.
2. 我注意到只有在model.compile()中使用run_eagerly=True时，我的模型才能工作.否则，我将收到此错误:"ValueError:尺寸必须相等，但对于...是1和2".我不确定为什么会出现这种情况，但是错误源于我使用tf.unsorted_segment_sum的那一行.
3. unique_with_counts_v2在tensorflow API中实际上并不存在，但是在源代码中存在.我需要能够将多个条件分组(而不是单个条件).

1. Because the last layer of my model is a sigmoid, my y_pred values are probabilities between 0 and 1. So I needed to convert them to 0s and 1s in order to calculate the ratios I need in my custom loss. At first I tried using tf.round, but I realized that is not differentiable. So instead I replaced it with y_pred = tf.sigmoid((y_pred - 0.5) * 1000) inside of custom_loss. This essentially takes all the y_pred values to 0 and 1, but in a differentiable way. It seems like a bit of a "hack" though, so please let me know if you have any feedback on this.
2. I noticed that my model only works if I use run_eagerly=True in model.compile(). Otherwise I get this error: "ValueError: Dimensions must be equal, but are 1 and 2 for ...". I'm not sure why this is the case, but the error originates from the line where I use tf.unsorted_segment_sum.
3. unique_with_counts_v2 does not actually exist in tensorflow API yet, but it exists in the source code. I needed this to be able to group by multiple conditions (not just a single one).

如果您对此有任何反馈，请随意发表评论，或者对上面的项目符号有回应.

Feel free to comment if you have any feedback on this, in general, or in response to the bullets above.

这篇关于每个张量组的Keras自定义损失函数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！