三重态损失的softmax版本的梯度计算 [英] Gradient calculation for softmax version of triplet loss

问题描述

我一直在尝试在Caffe中实现Caffe的三重态损失的softmax版本，
Hoffer和Ailon，使用三重态网络进行深度度量学习，ICLR 2015

I have been trying to implement the softmax version of the triplet loss in Caffe described in
Hoffer and Ailon, Deep Metric Learning Using Triplet Network, ICLR 2015.

我已经尝试过此方法，但是由于指数L2不成平方，因此我发现很难计算梯度.

I have tried this but I am finding it hard to calculate the gradient as the L2 in exponent is not squared.

有人可以在这里帮助我吗?

Can someone please help me here?

推荐答案

使用现有的caffe层来实施L2规范可以节省所有麻烦.

Implementing the L2 norm using existing layers of caffe can save you all the hustle.

这是在Caffe中针对底部" x1和x2计算||x1-x2||_2的一种方法(假设x1和x2是B -by- C斑点，计算B C尺寸差异的规范

Here's one way to compute ||x1-x2||_2 in caffe for "bottom"s x1 and x2 (assuming x1 and x2 are B-by-C blobs, computing B norms for C dimensional diffs)

layer {
  name: "x1-x2"
  type: "Eltwise"
  bottom: "x1"
  bottom: "x1"
  top: "x1-x2"
  eltwise_param { 
    operation: SUM
    coeff: 1 coeff: -1
  }
}
layer {
  name: "sqr_norm"
  type: "Reduction"
  bottom: "x1-x2"
  top: "sqr_norm"
  reduction_param { operation: SUMSQ axis: 1 }
}
layer {
  name: "sqrt"
  type: "Power"
  bottom: "sqr_norm"
  top: "sqrt"
  power_param { power: 0.5 }
}

对于本文中定义的三重态损失，您需要计算x-x+的L2范数，对于x-x-，将这两个斑点合并，并将concat斑点馈送到"Softmax"层.
无需进行肮脏的梯度计算.

For the triplet loss defined in the paper, you need to compute L2 norm for x-x+ and for x-x-, concat these two blobs and feed the concat blob to a "Softmax" layer.
No need for dirty gradient computations.

这篇关于三重态损失的softmax版本的梯度计算的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！