tensorflow是否像2D卷积一样允许LSTM解卷积(convlstm2d)? [英] Does tensorflow allow LSTM deconvolution ( convlstm2d) as it does for 2D convolution?

问题描述

我正在尝试扩大网络.对于卷积部分，我使用的是来自keras的convlstm2d.是否有执行反卷积的过程(即lstmdeconv2d?)

I am trying to augment a network. For the convolution part, I am using convlstm2d from keras. Is there a process to perform deconvolution ( i.e. lstmdeconv2d ? )

推荐答案

应该可以将任何模型与 TimeDistributed 包装器结合使用.因此，您可以创建一个deconv模型，并使用TimeDistributed包装器将其应用于LSTM的输出(向量序列).

It should be possible to combine any model with the TimeDistributed wrapper. So you can create a deconv-model, and apply it on the output (which is a sequence of vectors) of the LSTM using the TimeDistributed wrapper.

一个例子.首先使用Conv2DTranspose图层创建一个deconv网络.

An example. First create a deconv network using Conv2DTranspose layers.

from keras.models import Model
from keras.layers import LSTM,Conv2DTranspose, Input, Activation, Dense, Reshape, TimeDistributed

# Hyperparameters
layer_filters = [32, 64]

# Deconv Model 
# (adapted from https://github.com/keras-team/keras/blob/master/examples/mnist_denoising_autoencoder.py )

deconv_inputs = Input(shape=(lstm_dim,), name='deconv_input')
feature_map_shape = (None, 50, 50, 64) # deconvolve from [batch_size, 50,50,64] => [batch_size, 200,200,3]
x = Dense(feature_map_shape[1] * feature_map_shape[2] * feature_map_shape[3])(deconv_inputs)
x = Reshape((feature_map_shape[1], feature_map_shape[2],feature_map_shape[3]))(x)
for filters in layer_filters[::-1]:
   x = Conv2DTranspose(filters=16,kernel_size=3,strides=2,activation='relu',padding='same')(x)
x = Conv2DTranspose(filters=3,kernel_size=3, padding='same')(x) # last layer has 3 channels
deconv_output = Activation('sigmoid', name='deconv_output')(x)
deconv_model = Model(deconv_inputs, deconv_output, name='deconv_network')

然后，您可以使用TimeDistributed层将此deconv模型应用于LSTM的输出.

Then, you can apply this deconv-model to the outputs of your LSTM using the TimeDistributed layer.

# LSTM
lstm_input = Input(shape=(None,16), name='lstm_input') # => [batch_size, timesteps, input_dim]
lstm_outputs =  LSTM(units=64, return_sequences=True)(lstm_input) # => [batch_size, timesteps, output_dim]
predicted_images = TimeDistributed(deconv_model)(lstm_outputs)

model = Model(lstm_input , predicted_images , name='lstm_deconv')
model.summary()

这篇关于tensorflow是否像2D卷积一样允许LSTM解卷积(convlstm2d)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！