Tensorflow最大利润损失培训? [英] Tensorflow max-margin loss training?

问题描述

我想使用最大负损失函数在张量流中训练一个神经网络，每个正样本使用一个负样本:

I want to train a neural network in tensorflow with a max-margin loss function using one negative sample per positive sample:

max(0,1 -pos_score +neg_score)

我当前正在做的是这样的: 网络采用三个输入: input1 ，然后是一个正例 input2_pos 和一个负例 input2_neg . (这些是单词嵌入层的索引.)网络应该计算出一个分数，该分数表示两个示例之间的关联程度. 这是我的代码的简化版本:

What I'm currently doing is this: The network takes three inputs: input1, and then one positive example input2_pos and one negative example input2_neg. (These are indices to a word embeddings layer.) The network is supposed to calculate a score that expresses how related two examples are. Here's a simplified version of my code:

input1 = tf.placeholder(dtype=tf.int32, shape=[batch_size])
input2_pos = tf.placeholder(dtype=tf.int32, shape=[batch_size])
input2_neg = tf.placeholder(dtype=tf.int32, shape=[batch_size])

# f is a neural network outputting a score
pos_score = f(input1,input2_pos)
neg_score = f(input1,input2_neg)

cost = tf.maximum(0., 1. -pos_score +neg_score)
optimizer= tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)

运行此命令时，我看到的是，网络仅了解哪个输入具有正例-它总是按照以下方式预测相似的得分:

What I see when I run this, is that like this the network just learns which input holds the positive example - it always predicts a similar score along the lines of:

pos_score = 0.9965983
neg_score = 0.00341663

如何构造变量/训练，以便网络代替学习任务?

我只想要一个网络，它接受两个输入并计算一个表示它们之间相关性的分数，并以最大保证金损失进行训练.

I want just one network that takes two inputs and calculates a score expressing the correlation between them, and train it with max-margin loss.

对我来说，分别计算正面和负面的分数似乎不是一个选择，因为那样就不会正确地向后传播.另一个选择似乎是使输入随机化-但是对于损失函数，我需要知道哪个例子是正数-输入作为另一个参数的输入将再次放弃解决方案?

Calculating scores for positive and negative separately does not seem like an option to me, since then it won't backpropagate properly. Another option seems to be randomizing inputs - but then for the loss function I need to know which example is the positive one - inputting that as another parameter would give away the solution again?

有什么想法吗?

推荐答案

给出您的结果(每个正数为1，每个负数为0)似乎您在学习两个不同的网络:

Given your results (1 for every positive, 0 for every negative) it seems you have two different networks learning:

• 为第一个预测1
• 预测第二个为0

在使用最大保证金损失时，需要使用相同的网络来计算pos_score和neg_score.做到这一点的方法是共享变量.我会给你一个使用tf.get_variable()的小例子:

When using max-margin loss, you need to use the same network for computing both pos_score and neg_score. The way to do that is to share the variables. I will give you a small example using tf.get_variable():

with tf.variable_scope("network"):
    w = tf.get_variable("weights", shape=..., initializer=...)

def f(x, y):
    with tf.variable_scope("network", reuse=True):
        w = tf.get_variable("weights")
        res = w * (x - y)  # some computation
    return res

---

使用此功能f作为模型，训练将优化名称为网络/权重"的共享变量.

---

With this function f as model, the training will optimize the shared variable with name "network/weights".

这篇关于Tensorflow最大利润损失培训?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！