Tensorflow中bidirectional_dynamic_rnn和stack_bidirectional_dynamic_rnn的区别 [英] Difference between bidirectional_dynamic_rnn and stack_bidirectional_dynamic_rnn in Tensorflow

问题描述

我正在构建一个堆叠多个 LSTM 的动态 RNN 网络.我看到有 2 个选项

I am building a dynamic RNN network with stacking multiple LSTMs. I see there are 2 options

# cells_fw and cells_bw are list of cells eg LSTM cells
stacked_cell_fw = tf.contrib.rnn.MultiRNNCell(cells_fw)
stacked_cell_bw = tf.contrib.rnn.MultiRNNCell(cells_bw)

output = tf.nn.bidirectional_dynamic_rnn(
          stacked_cell_fw, stacked_cell_bw, INPUT,
          sequence_length=LENGTHS, dtype=tf.float32)

对比

output = tf.contrib.rnn.stack_bidirectional_dynamic_rnn(cells_fw, cells_bw, INPUT,
sequence_length=LENGTHS, dtype=tf.float32)

这两种方法有什么区别，一种比另一种更好吗?

What is the difference between the 2 approaches and is one better than the other?

推荐答案

如果您希望有多个层可以及时向后或向前传递信息，有两种设计方法.假设前向层由两层 F1、F2 组成，后向层由两层 B1、B2 组成.

If you want to have have multiple layers that pass the information backward or forward in time, there are two ways how to design this. Assume the forward layer consists of two layers F1, F2 and the backword layer consists of two layers B1, B2.

如果您使用 tf.nn.bidirectional_dynamic_rnn，模型将如下所示(时间从左到右):

If you use tf.nn.bidirectional_dynamic_rnn the model will look like this (time flows from left to right):

如果您使用 tf.contrib.rnn.stack_bidirectional_dynamic_rnn，模型将如下所示:

If you use tf.contrib.rnn.stack_bidirectional_dynamic_rnn the model will look like this:

此处第一层和第二层之间的黑点表示连接.即，前向和后向单元的输出连接在一起并馈送到下一个上层的后向和前向层.这意味着 F2 和 B2 接收完全相同的输入，并且后向层和前向层之间存在显式连接.在 深度循环神经网络的语音识别"中，Graves 等人.总结如下:

Here the black dot between first and second layer represents a concatentation. I.e., the outputs of the forward and backward cells are concatenated together and fed to the backward and forward layers of the next upper layer. This means both F2 and B2 receive exactly the same input and there is an explicit connection between backward and forward layers. In "Speech Recognition with Deep Recurrent Neural Networks" Graves et al. summarize this as follows:

... 每个隐藏层都接收来自下一层的前向和后向层.

... every hidden layer receives input from both the forward and backward layers at the level below.

此连接仅在未堆叠的 BiRNN(第一张图像)中隐式发生，即映射回输出时.对于我的目的，堆叠的 BiRNN 通常表现更好，但我想这取决于您的问题设置.但肯定值得一试！

This connection only happens implicitly in the unstacked BiRNN (first image), namely when mapping back to the output. The stacked BiRNN usually performed better for my purposes, but I guess that depends on your problem setting. But for sure it is worthwile to try it out!

编辑

回应你的评论:我的回答基于函数 tf.contrib.rnn.stack_bidirectional_dynamic_rnn 的文档，它说:

In response to your comment: I base my answer on the documentation of the function tf.contrib.rnn.stack_bidirectional_dynamic_rnn which says:

堆叠多个双向 rnn 层.联合前进和后向层输出用作下一层的输入.tf.bidirectional_rnn 不允许向前和向后共享层间信息.

Stacks several bidirectional rnn layers. The combined forward and backward layer outputs are used as input of the next layer. tf.bidirectional_rnn does not allow to share forward and backward information between layers.

另外，我查看了 此链接.

这篇关于Tensorflow中bidirectional_dynamic_rnn和stack_bidirectional_dynamic_rnn的区别的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！