在y_true和y_pred具有不同大小的keras中实现自定义损失功能 [英] Implementing custom loss function in keras with different sizes for y_true and y_pred

问题描述

我是Keras的新手.我需要一些帮助，使用TensorFlow后端在keras中为以下损失方程式编写自定义损失函数.

I am new to Keras. I need some help in writing a custom loss function in keras with TensorFlow backend for the following loss equation.

传递给损失函数的参数为​​:

The parameters passed to the loss function are :

1. y_true的形状为(batch_size, N, 2).在这里，我们在批次中的每个样本中传递N (x, y)坐标.
2. y_pred的形状为(batch_size, 256, 256, N).在这里，我们在批次中的每个样本中传递了256 x 256个像素的N个预测热图.

1. y_true would be of shape (batch_size, N, 2). Here, we are passing N (x, y) coordinates in each sample in the batch.
2. y_pred would be of shape (batch_size, 256, 256, N). Here, we are passing N predicted heatmaps of 256 x 256 pixels in each sample in the batch.

i∈ [0, 255]

j∈ [0, 255]

Mn(i, j)表示第n个 个预测热图在像素位置(i, j)处的值.

Mn(i, j) represents value at pixel location (i, j) for the n^th predicted heatmap.

Mn∼(i, j) = Guassian2D((i, j), y_truen, std)其中

Mn∼(i, j) = Guassian2D((i, j), y_truen, std) where

std = standard deviation，两个尺寸(5 px)的标准偏差都相同.

std = standard deviation, same standard deviation for both the dimensions (5 px).

y_true _n 是第n ^th (x，y)坐标.这是平均值.

y_true_n is the n^th (x, y) coordinate. This is the mean.

有关此详细信息，请检查本文所述的l ₂ 损失 人类姿势估计.

For details of this, please check the l₂ loss described in this paper Human Pose Estimation.

注意:我提到的batch_size形状为y_true和y_pred.我假设Keras在整个批次上调用损失函数，而不是在批次中的单个样本上调用损失函数.如果我错了，请纠正我.

Note : I mentioned batch_size in shape of y_true and y_pred. I assumed that Keras calls loss function on the entire batch and not on individual samples in the batch. Correct me if I am wrong.

def l2_loss(y_true, y_pred):
     loss = 0
     n = y_true.shape[0]
     for j in range(n):
        for i in range(num_joints):
            yv, xv = tf.meshgrid(tf.arange(0, im_height), tf.arange(0, im_width))
            z = np.array([xv, yv]).transpose(1, 2, 0)
            ground = np.exp(-0.5*(((z - y_true[j, i, :])**2).sum(axis=2))/(sigma**2))
            loss = loss + np.sum((ground - y_pred[j,:, :, i])**2)
     return loss/num_joints

这是我到目前为止编写的代码.我知道这不会运行，因为我们不能在keras损失函数中使用直接的numpy ndarrays.另外，我需要消除循环！

This is the code I have writen so far. I know that this won't run as we can't use direct numpy ndarrays inside a keras loss function. Also, I need to eliminate loops!

推荐答案

您几乎可以将numpy函数转换为Keras后端函数.唯一需要注意的是设置正确的广播形状.

You can pretty much just translate the numpy functions into Keras backend functions. The only thing to notice is to set up the right broadcast shape.

def l2_loss_keras(y_true, y_pred):
    # set up meshgrid: (height, width, 2)
    meshgrid = K.tf.meshgrid(K.arange(im_height), K.arange(im_width))
    meshgrid = K.cast(K.transpose(K.stack(meshgrid)), K.floatx())

    # set up broadcast shape: (batch_size, height, width, num_joints, 2)
    meshgrid_broadcast = K.expand_dims(K.expand_dims(meshgrid, 0), -2)
    y_true_broadcast = K.expand_dims(K.expand_dims(y_true, 1), 2)
    diff = meshgrid_broadcast - y_true_broadcast

    # compute loss: first sum over (height, width), then take average over num_joints
    ground = K.exp(-0.5 * K.sum(K.square(diff), axis=-1) / sigma ** 2)
    loss = K.sum(K.square(ground - y_pred), axis=[1, 2])
    return K.mean(loss, axis=-1)

要验证:

def l2_loss_numpy(y_true, y_pred):
     loss = 0
     n = y_true.shape[0]
     for j in range(n):
        for i in range(num_joints):
            yv, xv = np.meshgrid(np.arange(0, im_height), np.arange(0, im_width))
            z = np.stack([xv, yv]).transpose(1, 2, 0)
            ground = np.exp(-0.5*(((z - y_true[j, i, :])**2).sum(axis=2))/(sigma**2))
            loss = loss + np.sum((ground - y_pred[j,:, :, i])**2)
     return loss/num_joints

batch_size = 32
num_joints = 10
sigma = 5
im_width = 256
im_height = 256

y_true = 255 * np.random.rand(batch_size, num_joints, 2)
y_pred = 255 * np.random.rand(batch_size, im_height, im_width, num_joints)

print(l2_loss_numpy(y_true, y_pred))
45448272129.0

print(K.eval(l2_loss_keras(K.variable(y_true), K.variable(y_pred))).sum())
4.5448e+10

该数字在默认的dtype float32下被截断.如果将dtype设置为float64来运行它:

The number is truncated under the default dtype float32. If you run it with dtype set to float64:

y_true = 255 * np.random.rand(batch_size, num_joints, 2)
y_pred = 255 * np.random.rand(batch_size, im_height, im_width, num_joints)

print(l2_loss_numpy(y_true, y_pred))
45460126940.6

print(K.eval(l2_loss_keras(K.variable(y_true), K.variable(y_pred))).sum())
45460126940.6

---

看来Keras要求y_true和y_pred具有相同的尺寸数.例如，在以下测试模型上:

It seems that Keras requires y_true and y_pred to have the same number of dimensions. For example, on the following testing model:

X = np.random.rand(batch_size, 256, 256, 3)
model = Sequential([Dense(10, input_shape=(256, 256, 3))])
model.compile(loss=l2_loss_keras, optimizer='adam')
model.fit(X, y_true, batch_size=8)

ValueError: Cannot feed value of shape (8, 10, 2) for Tensor 'dense_2_target:0', which has shape '(?, ?, ?, ?)'

要解决此问题，您可以在将y_true输入模型之前，在expand_dims中添加一个虚拟尺寸:

To deal with this problem, you can add a dummy dimension with expand_dims before feeding y_true into the model:

def l2_loss_keras(y_true, y_pred):
    ...

    y_true_broadcast = K.expand_dims(y_true, 1)  # change this line

    ...

model.fit(X, np.expand_dims(y_true, axis=1), batch_size=8)

这篇关于在y_true和y_pred具有不同大小的keras中实现自定义损失功能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！