TensorFlow中的最大保证金损失 [英] Max margin loss in TensorFlow

问题描述

我正在尝试在TensorFlow中实现最大利润损失. 我的想法是，我有一些正面的例子，而我对一些负面的例子进行了采样，并希望计算出类似的值

I'm trying to implement a max margin loss in TensorFlow. the idea is that I have some positive example and i sample some negative examples and want to compute something like

其中B是我的批量大小，N是我要使用的阴性样品的数量.

where B is the size of my batch and N is the number of negative samples I want to use.

我是tensorflow的新手，我发现实现它很棘手. 我的模型计算了一个维度为B * (N + 1)的分数向量，其中我交替使用正样本和负样本.例如，对于批次大小为2和2的负面示例，我有一个大小为6的向量，其中第一个正面示例在索引0处的得分，第二正面示例在位置3处的得分，而负面示例在位置1、2处的得分4和5. 理想的方法是获取[1, 0, 0, 1, 0, 0]之类的值.

I'm new to tensorflow and I'm finding it tricky to implement it. My model computes a vector of scores of dimension B * (N + 1) where I alternate positive samples and negative samples. For instance, for a batch size of 2 and 2 negative examples I have a vector of size 6 with scores for the first positive example at index 0 and for the second positive example at position 3 and scores for negative examples in position 1, 2, 4 and 5. The ideal would be to get values like [1, 0, 0, 1, 0, 0].

使用while和条件，我可以想到的是以下内容:

What I could came up with is the following, using while and conditions:

# Function for computing max margin inner loop
def max_margin_inner(i, batch_examples_t, j, scores, loss):
    idx_pos = tf.mul(i, batch_examples_t)
    score_pos = tf.gather(scores, idx_pos)
    idx_neg = tf.add_n([tf.mul(i, batch_examples_t), j, 1])
    score_neg = tf.gather(scores, idx_neg)
    loss = tf.add(loss, tf.maximum(0.0, 1.0 - score_pos + score_neg))
    tf.add(j, 1)
    return [i, batch_examples_t, j, scores, loss]

# Function for computing max margin outer loop
def max_margin_outer(i, batch_examples_t, scores, loss):
    j = tf.constant(0)
    pos_idx = tf.mul(i, batch_examples_t)
    length = tf.gather(tf.shape(scores), 0)
    neg_smp_t = tf.constant(num_negative_samples)
    cond = lambda i, b, j, bi, lo: tf.logical_and(
        tf.less(j, neg_smp_t),
        tf.less(pos_idx, length))
    tf.while_loop(cond, max_margin_inner, [i, batch_examples_t, j, scores, loss])
    tf.add(i, 1)
    return [i, batch_examples_t, scores, loss]

# compute the loss
with tf.name_scope('max_margin'):
    loss = tf.Variable(0.0, name="loss")
    i = tf.constant(0)
    batch_examples_t = tf.constant(batch_examples)
    condition = lambda i, b, bi, lo: tf.less(i, b)
    max_margin = tf.while_loop(
        condition,
        max_margin_outer,
        [i, batch_examples_t, scores, loss])

该代码有两个循环，一个循环用于外部和，另一个循环用于内部.我面临的问题是，损失变量在每次迭代中都会不断累积错误，而在每次迭代之后都不会重置.因此，它实际上根本不起作用.

The code has two loops, one for the outer sum and the other for the inner one. The problem I'm facing is that the loss variable keeps accumulating errors at each iteration without being reset after each iteration. So it actually doesn't work at all.

此外，似乎真的不符合实现事物的张量流方式.我猜可能会有更好的方法，更多的矢量化方法来实现它，希望有人会提出建议或为我提供示例.

Moreover, it seems really not in line with tensorflow way of implementing things. I guess there could be better ways, more vectorized ways to implement it, hope someone will suggest options or point me to examples.

推荐答案

首先，我们需要清除输入内容:

First we need to clean the input:

• 我们想要一个形状为[B, 1]
的正分数数组
• 我们想要一个负得分矩阵，形状为[B, N]

• we want an array of positive scores, of shape [B, 1]
• we want a matrix of negative scores, of shape [B, N]

import tensorflow as tf

B = 2
N = 2
scores = tf.constant([0.5, 0.2, -0.1, 1., -0.5, 0.3])  # shape B * (N+1)

scores = tf.reshape(scores, [B, N+1])

scores_pos = tf.slice(scores, [0, 0], [B, 1])

scores_neg = tf.slice(scores, [0, 1], [B, N])

---

现在，我们只需要计算损失矩阵，即每对损失(正负)的所有单个损失，并计算其总和即可.

---

Now we only have to compute the matrix of the loss, i.e. all the individual loss for every pair (positive, negative), and compute its sum.

loss_matrix = tf.maximum(0., 1. - scores_pos + scores_neg)  # we could also use tf.nn.relu here
loss = tf.reduce_sum(loss_matrix)

这篇关于TensorFlow中的最大保证金损失的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！