keras中的可变长度输出 [英] Variable length output in keras

查看:17
本文介绍了keras中的可变长度输出的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我正在尝试在 keras 中创建一个带有分桶的自动编码器,其中输入和输出具有不同的时间步长.

I'm trying to create an autoencoder in keras with bucketing where the input and the output have different time steps.

model = Sequential()

#encoder
model.add(Embedding(vocab_size, embedding_size, mask_zero=True))
model.add(LSTM(units=hidden_size, return_sequences=False))

#decoder
model.add(RepeatVector(max_out_length))
model.add(LSTM(units=hidden_size, return_sequences=True))
model.add(TimeDistributed(Dense(num_class, activation='softmax')))

对于输入没有问题,因为只要整个批次具有相同的长度,网络就可以接受不同长度的输入.然而,问题在于输出大小由 RepeatVector 长度决定,并且没有简单的方法可以改变它.

For the input there is no problem as the network can accept different length inputs as long as the whole batch has the same length. However the problem is with the output size as its determined by the RepeatVector length and there is not easy way to change it.

有没有办法解决这样的问题?

Is there a solution for such a problem?

推荐答案

如果你的意思是输入长度可变"和输出长度与输入相同",你可以这样做:

If you mean "inputs with variable lengths" and "outputs with the same lengths as the inputs", you can do this:

警告:此解决方案必须适用于批量大小 = 1
您将需要创建一个外部循环并将每个样本作为具有精确长度的 numpy 数组传递
您不能在此解决方案中使用掩码,正确的输出取决于输入的正确长度

Warning: this solution must work with batch size = 1
You will need to create an external loop and pass each sample as a numpy array with the exact length
You cannot use masking in this solution, and the right output depends on the correct length of the input

这是使用 Keras + Tensorflow 的工作代码:

进口:

from keras.layers import *
from keras.models import Model
import numpy as np
import keras.backend as K
from keras.utils.np_utils import to_categorical

要在 Lambda 层中使用的自定义函数:

Custom functions to use in Lambda layers:

#this function gets the length from the original input 
#and stores it in the final output of the encoder
def storeLength(x):
    inputTensor = x[0]
    storeInto = x[1] #the final output

    length = K.shape(inputTensor)[1]
    length = K.cast(length,K.floatx())
    length = K.reshape(length,(1,1))

    #will put length as the first element in the final output
    return K.concatenate([length,storeInto])


#this function expands the length of the input in the decoder
def expandLength(x):
    #lenght is the first element in the encoded input
    length = K.cast(x[0,0],'int32') #or int64 if necessary

    #the remaining elements are the actual data to be decoded
    data = x[:,1:]

    #a tensor with shape (length,)
    length = K.ones_like(K.arange(0,length))

    #make both length tensor and data tensor 3D and with paired dimensions 
    length = K.cast(K.reshape(length,(1,-1,1)),K.floatx())
    data = K.reshape(data,(1,1,-1))

    #this automatically repeats the elements based on the paired shapes
    return data*length 

创建模型:

我假设输出等于输入,但由于您使用的是嵌入,我将num_classes"设置为等于字数.

I assumed the output is equal to the input, but since you're using an Embedding, I made "num_classes" equal to the number of words.

对于这个解决方案,我们使用了一个分支,因此我不得不使用函数式 API Model.稍后会更好,因为您需要使用 autoencoder.train_on_batch 进行训练,然后使用 encoder.predict() 进行编码或使用 decoder 进行解码.预测().

For this solution, we use a branching, thus I had to use the functional API Model. Which will be way better later, because you will want to train with autoencoder.train_on_batch and then just encode with encoder.predict() or just decode with decoder.predict().

vocab_size = 100
embedding_size = 7
num_class=vocab_size
hidden_size = 3

#encoder
inputs = Input(batch_shape = (1,None))
outputs = Embedding(vocab_size, embedding_size)(inputs)
outputs = LSTM(units=hidden_size, return_sequences=False)(outputs)
outputs = Lambda(storeLength)([inputs,outputs])
encoder = Model(inputs,outputs)

#decoder
inputs = Input(batch_shape=(1,hidden_size+1))
outputs = Lambda(expandLength)(inputs)
outputs = LSTM(units=hidden_size, return_sequences=True)(outputs)
outputs = TimeDistributed(Dense(num_class, activation='softmax'))(outputs)
decoder = Model(inputs,outputs)

#autoencoder
inputs = Input(batch_shape=(1,None))
outputs = encoder(inputs)
outputs = decoder(outputs)
autoencoder = Model(inputs,outputs)

#see each model's shapes 
encoder.summary()
decoder.summary()
autoencoder.summary()

只是一个假数据的例子和应该用于训练的方法:

Just an example with fake data and the method that should be used for training:

inputData = []
outputData = []
for i in range(7,10):
    inp = np.arange(i).reshape((1,i))
    inputData.append(inp)

    outputData.append(to_categorical(inp,num_class))

autoencoder.compile(loss='mse',optimizer='adam')

for epoch in range(1):
    for inputSample,outputSample in zip(inputData,outputData):

        print(inputSample.shape,outputSample.shape)
        autoencoder.train_on_batch(inputSample,outputSample)

for inputSample in inputData:
    print(autoencoder.predict(inputSample).shape)

这篇关于keras中的可变长度输出的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆