keras中的可变长度输出 [英] Variable length output in keras
问题描述
我正在尝试在 keras 中创建一个带有分桶的自动编码器,其中输入和输出具有不同的时间步长.
I'm trying to create an autoencoder in keras with bucketing where the input and the output have different time steps.
model = Sequential()
#encoder
model.add(Embedding(vocab_size, embedding_size, mask_zero=True))
model.add(LSTM(units=hidden_size, return_sequences=False))
#decoder
model.add(RepeatVector(max_out_length))
model.add(LSTM(units=hidden_size, return_sequences=True))
model.add(TimeDistributed(Dense(num_class, activation='softmax')))
对于输入没有问题,因为只要整个批次具有相同的长度,网络就可以接受不同长度的输入.然而,问题在于输出大小由 RepeatVector 长度决定,并且没有简单的方法可以改变它.
For the input there is no problem as the network can accept different length inputs as long as the whole batch has the same length. However the problem is with the output size as its determined by the RepeatVector length and there is not easy way to change it.
有没有办法解决这样的问题?
Is there a solution for such a problem?
推荐答案
如果你的意思是输入长度可变"和输出长度与输入相同",你可以这样做:
If you mean "inputs with variable lengths" and "outputs with the same lengths as the inputs", you can do this:
警告:此解决方案必须适用于批量大小 = 1
您将需要创建一个外部循环并将每个样本作为具有精确长度的 numpy 数组传递
您不能在此解决方案中使用掩码,正确的输出取决于输入的正确长度
Warning: this solution must work with batch size = 1
You will need to create an external loop and pass each sample as a numpy array with the exact length
You cannot use masking in this solution, and the right output depends on the correct length of the input
这是使用 Keras + Tensorflow 的工作代码:
进口:
from keras.layers import *
from keras.models import Model
import numpy as np
import keras.backend as K
from keras.utils.np_utils import to_categorical
要在 Lambda 层中使用的自定义函数:
Custom functions to use in Lambda layers:
#this function gets the length from the original input
#and stores it in the final output of the encoder
def storeLength(x):
inputTensor = x[0]
storeInto = x[1] #the final output
length = K.shape(inputTensor)[1]
length = K.cast(length,K.floatx())
length = K.reshape(length,(1,1))
#will put length as the first element in the final output
return K.concatenate([length,storeInto])
#this function expands the length of the input in the decoder
def expandLength(x):
#lenght is the first element in the encoded input
length = K.cast(x[0,0],'int32') #or int64 if necessary
#the remaining elements are the actual data to be decoded
data = x[:,1:]
#a tensor with shape (length,)
length = K.ones_like(K.arange(0,length))
#make both length tensor and data tensor 3D and with paired dimensions
length = K.cast(K.reshape(length,(1,-1,1)),K.floatx())
data = K.reshape(data,(1,1,-1))
#this automatically repeats the elements based on the paired shapes
return data*length
创建模型:
我假设输出等于输入,但由于您使用的是嵌入,我将num_classes"设置为等于字数.
I assumed the output is equal to the input, but since you're using an Embedding, I made "num_classes" equal to the number of words.
对于这个解决方案,我们使用了一个分支,因此我不得不使用函数式 API Model
.稍后会更好,因为您需要使用 autoencoder.train_on_batch
进行训练,然后使用 encoder.predict()
进行编码或使用 decoder 进行解码.预测()
.
For this solution, we use a branching, thus I had to use the functional API Model
. Which will be way better later, because you will want to train with autoencoder.train_on_batch
and then just encode with encoder.predict()
or just decode with decoder.predict()
.
vocab_size = 100
embedding_size = 7
num_class=vocab_size
hidden_size = 3
#encoder
inputs = Input(batch_shape = (1,None))
outputs = Embedding(vocab_size, embedding_size)(inputs)
outputs = LSTM(units=hidden_size, return_sequences=False)(outputs)
outputs = Lambda(storeLength)([inputs,outputs])
encoder = Model(inputs,outputs)
#decoder
inputs = Input(batch_shape=(1,hidden_size+1))
outputs = Lambda(expandLength)(inputs)
outputs = LSTM(units=hidden_size, return_sequences=True)(outputs)
outputs = TimeDistributed(Dense(num_class, activation='softmax'))(outputs)
decoder = Model(inputs,outputs)
#autoencoder
inputs = Input(batch_shape=(1,None))
outputs = encoder(inputs)
outputs = decoder(outputs)
autoencoder = Model(inputs,outputs)
#see each model's shapes
encoder.summary()
decoder.summary()
autoencoder.summary()
只是一个假数据的例子和应该用于训练的方法:
Just an example with fake data and the method that should be used for training:
inputData = []
outputData = []
for i in range(7,10):
inp = np.arange(i).reshape((1,i))
inputData.append(inp)
outputData.append(to_categorical(inp,num_class))
autoencoder.compile(loss='mse',optimizer='adam')
for epoch in range(1):
for inputSample,outputSample in zip(inputData,outputData):
print(inputSample.shape,outputSample.shape)
autoencoder.train_on_batch(inputSample,outputSample)
for inputSample in inputData:
print(autoencoder.predict(inputSample).shape)
这篇关于keras中的可变长度输出的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!