在输入时我只有两个文件(即在测试时)如何处理三重态丢失 [英] How to deal with triplet loss when at time of input i have only two files i.e. at time of testing

问题描述

我正在实现一个暹罗网络，该网络中我知道如何通过将输入分为三个部分(这是一个手工制作的特征向量)来选择正锚和负锚来计算三重态损失，然后在训练时进行计算.

anchor_output = ...  # shape [None, 128]
positive_output = ...  # shape [None, 128]
negative_output = ...  # shape [None, 128]

d_pos = tf.reduce_sum(tf.square(anchor_output - positive_output), 1)
d_neg = tf.reduce_sum(tf.square(anchor_output - negative_output), 1)

loss = tf.maximum(0., margin + d_pos - d_neg)
loss = tf.reduce_mean(loss)

但是问题是，在测试时，我只有两个正负文件，然后我将如何处理(三胞胎，因为我还需要一个锚文件，但我的应用仅拍一张照片并与数据库进行比较因此在这种情况下只有两个文件)，我进行了大量搜索，但没有人提供代码来解决此问题，只是存在实现三重态丢失的代码，但没有针对整个场景. 而且我不想使用对比损失

解决方案

在CIFAR 10上具有测试代码的Colab笔记本: https://colab.research.google.com/drive/1VgOTzr_VZNHkXh2z9IiTAcEgg5qr19> >

总体思路:

from tensorflow import keras
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model
from tensorflow.keras import backend as K

img_width = 128
img_height = 128
img_colors = 3

margin = 1.0

VECTOR_SIZE = 32

def triplet_loss(y_true, y_pred):
  """ y_true is a dummy value that should be ignored

      Uses the inverse of the cosine similarity as a loss.
  """
  anchor_vec = y_pred[:, :VECTOR_SIZE]
  positive_vec = y_pred[:, VECTOR_SIZE:2*VECTOR_SIZE]
  negative_vec = y_pred[:, 2*VECTOR_SIZE:]
  d1 = keras.losses.cosine_proximity(anchor_vec, positive_vec)
  d2 = keras.losses.cosine_proximity(anchor_vec, negative_vec)
  return K.clip(d2 - d1 + margin, 0, None)


def make_image_model():
  """ Build a convolutional model that generates a vector
  """
  inp = Input(shape=(img_width, img_height, img_colors))
  l1 = Conv2D(8, (2, 2))(inp)
  l1 = MaxPooling2D()(l1)
  l2 = Conv2D(16, (2, 2))(l1)
  l2 = MaxPooling2D()(l2)
  l3 = Conv2D(16, (2, 2))(l2)
  l3 = MaxPooling2D()(l3)
  conv_out = Flatten()(l3)
  out = Dense(VECTOR_SIZE)(conv_out)
  model = Model(inp, out)
  return model

def make_siamese_model(img_model):
  """ Siamese model input are 3 images base, positive, negative
      output is a dummy variable that is ignored for the purposes of loss
      calculation.
  """
  anchor = Input(shape=(img_width, img_height, img_colors))
  positive = Input(shape=(img_width, img_height, img_colors))
  negative = Input(shape=(img_width, img_height, img_colors))
  anchor_vec = img_model(anchor)
  positive_vec = img_model(positive)
  negative_vec = img_model(negative)
  vecs = Concatenate(axis=1)([anchor_vec, positive_vec, negative_vec])
  model = Model([anchor, positive, negative], vecs)
  model.compile('adam', triplet_loss)
  return model

img_model = make_image_model()
train_model = make_siamese_model(img_model)
img_model.summary()
train_model.summary()

###
train_model.fit(X, dummy_y, ...)

img_model.save('image_model.h5')

###
# In order to use the model
vec_base = img_model.predict(base_image)
vec_test = img_model.predict(test_image)

比较vec_base和vec_test的余弦相似度，以确定碱基和检验是否在可接受的标准之内.

I am implementing a siamese network in which i know how to calculate triplet loss by picking anchor, positive and negative by dividing input in three parts(which is a handcrafted feature vector) and then calculating it at time of training.

anchor_output = ...  # shape [None, 128]
positive_output = ...  # shape [None, 128]
negative_output = ...  # shape [None, 128]

d_pos = tf.reduce_sum(tf.square(anchor_output - positive_output), 1)
d_neg = tf.reduce_sum(tf.square(anchor_output - negative_output), 1)

loss = tf.maximum(0., margin + d_pos - d_neg)
loss = tf.reduce_mean(loss)

But the problem is when at time of testing i would be having only two files positive and negative then how i would deal with(triplets, as i need one more anchor file but my app only take one picture and compare with in database so only two files in this case), I searched a lot but nobody provided code to deal with this problem only there was code to implement triplet loss but not for whole scenario. AND I DONT WANT TO USE CONTRASTIVE LOSS

解决方案

Colab notebook with test code on CIFAR 10: https://colab.research.google.com/drive/1VgOTzr_VZNHkXh2z9IiTAcEgg5qr19y0

The general idea:

from tensorflow import keras
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model
from tensorflow.keras import backend as K

img_width = 128
img_height = 128
img_colors = 3

margin = 1.0

VECTOR_SIZE = 32

def triplet_loss(y_true, y_pred):
  """ y_true is a dummy value that should be ignored

      Uses the inverse of the cosine similarity as a loss.
  """
  anchor_vec = y_pred[:, :VECTOR_SIZE]
  positive_vec = y_pred[:, VECTOR_SIZE:2*VECTOR_SIZE]
  negative_vec = y_pred[:, 2*VECTOR_SIZE:]
  d1 = keras.losses.cosine_proximity(anchor_vec, positive_vec)
  d2 = keras.losses.cosine_proximity(anchor_vec, negative_vec)
  return K.clip(d2 - d1 + margin, 0, None)


def make_image_model():
  """ Build a convolutional model that generates a vector
  """
  inp = Input(shape=(img_width, img_height, img_colors))
  l1 = Conv2D(8, (2, 2))(inp)
  l1 = MaxPooling2D()(l1)
  l2 = Conv2D(16, (2, 2))(l1)
  l2 = MaxPooling2D()(l2)
  l3 = Conv2D(16, (2, 2))(l2)
  l3 = MaxPooling2D()(l3)
  conv_out = Flatten()(l3)
  out = Dense(VECTOR_SIZE)(conv_out)
  model = Model(inp, out)
  return model

def make_siamese_model(img_model):
  """ Siamese model input are 3 images base, positive, negative
      output is a dummy variable that is ignored for the purposes of loss
      calculation.
  """
  anchor = Input(shape=(img_width, img_height, img_colors))
  positive = Input(shape=(img_width, img_height, img_colors))
  negative = Input(shape=(img_width, img_height, img_colors))
  anchor_vec = img_model(anchor)
  positive_vec = img_model(positive)
  negative_vec = img_model(negative)
  vecs = Concatenate(axis=1)([anchor_vec, positive_vec, negative_vec])
  model = Model([anchor, positive, negative], vecs)
  model.compile('adam', triplet_loss)
  return model

img_model = make_image_model()
train_model = make_siamese_model(img_model)
img_model.summary()
train_model.summary()

###
train_model.fit(X, dummy_y, ...)

img_model.save('image_model.h5')

###
# In order to use the model
vec_base = img_model.predict(base_image)
vec_test = img_model.predict(test_image)

compare cosine similarity of vec_base and vec_test in order to determine whether base and test are within the acceptable criteria.

这篇关于在输入时我只有两个文件(即在测试时)如何处理三重态丢失的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！