softmax 版本的triplet loss 的梯度计算 [英] Gradient calculation for softmax version of triplet loss

问题描述

我一直在尝试在
中描述的Caffe中实现三元组损失的softmax版本Hoffer 和 Ailon，使用三元组网络的深度度量学习，ICLR 2015.

I have been trying to implement the softmax version of the triplet loss in Caffe described in
Hoffer and Ailon, Deep Metric Learning Using Triplet Network, ICLR 2015.

我已经尝试过这个，但我发现很难计算梯度，因为指数中的 L2 不是平方.

I have tried this but I am finding it hard to calculate the gradient as the L2 in exponent is not squared.

有人可以帮我吗?

推荐答案

使用现有的 caffe 层实现 L2 规范可以为您省去所有麻烦.

Implementing the L2 norm using existing layers of caffe can save you all the hustle.

这是在 caffe 中为bottom"s x1 和 x2 计算 ||x1-x2||_2 的一种方法(假设 x1 和 x2 是 B-by-C blobs，为 B 计算范数code>C 维度差异)

Here's one way to compute ||x1-x2||_2 in caffe for "bottom"s x1 and x2 (assuming x1 and x2 are B-by-C blobs, computing B norms for C dimensional diffs)

layer {
  name: "x1-x2"
  type: "Eltwise"
  bottom: "x1"
  bottom: "x1"
  top: "x1-x2"
  eltwise_param { 
    operation: SUM
    coeff: 1 coeff: -1
  }
}
layer {
  name: "sqr_norm"
  type: "Reduction"
  bottom: "x1-x2"
  top: "sqr_norm"
  reduction_param { operation: SUMSQ axis: 1 }
}
layer {
  name: "sqrt"
  type: "Power"
  bottom: "sqr_norm"
  top: "sqrt"
  power_param { power: 0.5 }
}

对于论文中定义的triplet loss，你需要计算x-x+和xx-的L2范数，连接这两个blob并输入concat blob到 Softmax" 层.
不需要脏梯度计算.

For the triplet loss defined in the paper, you need to compute L2 norm for x-x+ and for x-x-, concat these two blobs and feed the concat blob to a "Softmax" layer.
No need for dirty gradient computations.

这篇关于softmax 版本的triplet loss 的梯度计算的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！