如何在TensorFlow中处理具有可变长度序列的批处理? [英] How to deal with batches with variable-length sequences in TensorFlow?

问题描述

我正在尝试使用RNN(特别是LSTM)进行序列预测.但是，我遇到了序列长度可变的问题.例如，

I was trying to use an RNN (specifically, LSTM) for sequence prediction. However, I ran into an issue with variable sequence lengths. For example,

sent_1 = "I am flying to Dubain"
sent_2 = "I was traveling from US to Dubai"

我试图基于此但是，num_steps参数(用于展开到先前的隐藏状态)在每个Tensorflow的时期应保持相同.基本上，批处理句子是不可能的，因为句子的长度会有所不同.

However, the num_steps parameter (used for unrolling to the previous hidden states), should remain the same in each Tensorflow's epoch. Basically, batching sentences is not possible as the sentences vary in length.

 # inputs = [tf.squeeze(input_, [1])
 #           for input_ in tf.split(1, num_steps, inputs)]
 # outputs, states = rnn.rnn(cell, inputs, initial_state=self._initial_state)

在这里，对于每个句子，在我的情况下都需要更改num_steps.我已经尝试了几种方法，但是似乎没有任何效果.

Here, num_steps need to be changed in my case for every sentence. I have tried several hacks, but nothing seems working.

推荐答案

您可以使用以下介绍的存储桶和填充概念:

You can use the ideas of bucketing and padding which are described in:

    序列到序列模型

    Sequence-to-Sequence Models

此外，创建RNN网络的rnn函数接受参数 sequence_length.

Also, the rnn function which creates RNN network accepts parameter sequence_length.

例如，您可以创建相同大小的句子存储桶，将它们填充必要的零，或使用占位符表示零个单词，然后将它们与seq_length = len(zero_words)一起提供.

As an example, you can create buckets of sentences of the same size, pad them with the necessary amount of zeros, or placeholders which stand for zero word and afterwards feed them along with seq_length = len(zero_words).

seq_length = tf.placeholder(tf.int32)
outputs, states = rnn.rnn(cell, inputs, initial_state=initial_state, sequence_length=seq_length)

sess = tf.Session()
feed = {
    seq_length: 20,
    #other feeds
}
sess.run(outputs, feed_dict=feed)

也看一下这个reddit线程:

Take a look at this reddit thread as well:

   带有可变长度"的Tensorflow基本RNN示例序列

   Tensorflow basic RNN example with 'variable length' sequences

这篇关于如何在TensorFlow中处理具有可变长度序列的批处理?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！