在张量流中实现连体网络 [英] Implementing a Siamese Network in Tensor Flow

问题描述

我想实现一个连体卷积神经网络，其中两个图像在卷积层中共享权重，然后在通过全连接层之前进行连接.我尝试了一个实现，但它似乎是一个被黑"的解决方案.特别是，我将张量上的操作定义为简单的 Python 函数，我不确定是否允许这样做.

I want to implement a Siamese Convolutional Neural Network, where two images share weights in the convolutional layers, and are then concatenated before being passed through the fully-connected layers. I have tried an implementation, but it seems rather a "hacked" solution. In particular, I have defined an operation on tensors as simply a Python function, and I'm not sure whether this is allowed.

这是我试过的代码:

images = tf.placeholder(tf.float32, shape=[None, 64 * 64])
# Convolutional layers
# ...
# ...
# Results in pool3_flat, which is the flattened output of the third convolutional layer
pool3_flat = tf.reshape(pool3, [-1, 8 * 8 * 128])

# Now, merge the image pairs, where each pair is composed of adjacent images in the batch, with a stride of 2
def merge_pairs():
  # Create a tensor to store the merged image pairs
  # The batch size is 128, therefore there will be 64 pairs (64 in the first dimension of this tensor)
  merged_pairs = tf.Variable(tf.zeros([64, 8 * 8 * 128]))
  # Split the images into 64 pairs
  pairs = tf.split(0, 64, pool3_flat)
  # For each pair, concatenate the two images across dimension 1, and set this tensor in the appropriate row of merged_pairs
  for pair_num, pair in enumerate(pairs):
      merged_pair = tf.concat(1, pair)
      merged_pairs[pair_num] = merged_pair
  return merged_pairs


# Proceed with operations on the merged_pair tensor, as if the batch size is 64
fc4 = tf.matmul(merge_pairs(), weights4)
# ...
# ...

虽然这可以编译并且似乎运行良好，但结果并不像预期的那样.所以，我想知道是否有更好的方法来使用 TensorFlow 中的内置操作来实现 Siamese 网络?

Whilst this compiles and seems to run fine, the results are not really as expected. So, I'm wondering if there is a better way to implement a Siamese network using built-in operations in TensorFlow?

推荐答案

您可以使用 tf.pack和 tf.unpack，有点像:

You can make use of tf.pack and tf.unpack, somewhat like:

pairs = tf.pack(tf.split(0, 64, pool3_flat))
left, right = tf.unpack(tf.transpose(pairs, perm=[1,0,2]))
merged_pairs = tf.concat(1, [left, right])

一种更简洁的方法是从一开始就将您的配对分开，以便您可以定义两个网络并在每个网络中使用相同的可训练变量.

A cleaner way to do this is to keep your pairs separate from the beginning, so that you can define two networks and use the same trainable variables in each network.

你会有类似的东西(跳过卷积层):

You would have something like (skipping the convolutional layers):

image_left = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])
image_right = tf.placeholder(tf.float32, shape=[None, 64, 64, 1])

pool_left = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
pool_right = tf.nn.max_pool(image_left, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

pool_flat_left = tf.reshape(pool_left, [-1, 32*32])
pool_flat_right = tf.reshape(pool_right, [-1, 32*32])

然后简单地在维度 1 中左右连接.

Then simply concat left and right in dimension 1.

concat_layer = tf.concat(1, [pool_flat_left, pool_flat_right])

通过这种方式，您也可以稍后更改批量大小.确保对每个尺寸(左侧和右侧)使用相同的权重和偏差.

This way you can also vary the batch size later. Make sure to use the same weights and biases on each size (left and right).

这篇关于在张量流中实现连体网络的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！