Tensorflow:迷你批量中的每个样本使用不同过滤器的卷积 [英] Tensorflow: Convolutions with different filter for each sample in the mini-batch

问题描述

我想用一个二维卷积滤波器，该滤波器取决于张量流中小批处理中的样本.有什么想法可以做到的，特别是如果每​​个小批量的样品数量未知时?

I would like to have a 2d convolution with a filter which depends on the sample in the mini-batch in tensorflow. Any ideas how one could do that, especially if the number of sample per mini-batch is not known?

具体来说，我具有格式为MB x H x W x Channels的输入数据inp，并且具有格式为MB x fh x fw x Channels x OutChannels的过滤器F.

Concretely, I have input data inp of the form MB x H x W x Channels, and I have filters F of the form MB x fh x fw x Channels x OutChannels.

假设

inp = tf.placeholder('float', [None, H, W, channels_img], name='img_input').

我想做tf.nn.conv2d(inp, F, strides = [1,1,1,1])，但这是不允许的，因为F不能具有小批量尺寸.任何想法如何解决这个问题?

I would like to do tf.nn.conv2d(inp, F, strides = [1,1,1,1]), but this is not allowed because F cannot have a mini-batch dimension. Any idea how to solve this problem?

推荐答案

我认为建议的技巧实际上是不正确的. tf.conv3d()层会发生什么，就是将输入卷积到深度(=实际批次)维上，然后沿着结果要素图求和.使用padding='SAME'时，输出的结果数量恰好与批处理大小相同，因此被欺骗了！

I think the proposed trick is actually not right. What happens with a tf.conv3d() layer is that the input gets convolved on depth (=actual batch) dimension AND then summed along resulting feature maps. With padding='SAME' the resulting number of outputs then happens to be the same as batch size so one gets fooled!

我认为对不同的迷你批处理元素使用不同的滤镜进行卷积的一种可能方法涉及破解"深度卷积.假设批量大小MB是已知的:

I think a possible way to do a convolution with different filters for the different mini-batch elements involves 'hacking' a depthwise convolution. Assuming batch size MB is known:

inp = tf.placeholder(tf.float32, [MB, H, W, channels_img])

# F has shape (MB, fh, fw, channels, out_channels)
# REM: with the notation in the question, we need: channels_img==channels

F = tf.transpose(F, [1, 2, 0, 3, 4])
F = tf.reshape(F, [fh, fw, channels*MB, out_channels)

inp_r = tf.transpose(inp, [1, 2, 0, 3]) # shape (H, W, MB, channels_img)
inp_r = tf.reshape(inp, [1, H, W, MB*channels_img])

out = tf.nn.depthwise_conv2d(
          inp_r,
          filter=F,
          strides=[1, 1, 1, 1],
          padding='VALID') # here no requirement about padding being 'VALID', use whatever you want. 
# Now out shape is (1, H, W, MB*channels*out_channels)

out = tf.reshape(out, [H, W, MB, channels, out_channels) # careful about the order of depthwise conv out_channels!
out = tf.transpose(out, [2, 0, 1, 3, 4])
out = tf.reduce_sum(out, axis=3)

# out shape is now (MB, H, W, out_channels)

如果MB未知，应该可以使用tf.shape()(我认为)动态确定它

In case MB is unknown, it should be possible to determine it dynamically using tf.shape() (I think)

这篇关于Tensorflow:迷你批量中的每个样本使用不同过滤器的卷积的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！