如何在 Tensorflow 中实现堆叠的 RNN? [英] How to implement a stacked RNNs in Tensorflow?

问题描述

我想在 GPU 上使用 Tensorflow1.13 实现 RNN.按照官方的推荐，我写了下面的代码，得到一堆RNN cell

lstm = [tk.layers.CuDNNLSTM(128) for _ in range(2)]细胞 = tk.layers.StackedRNNCells(lstm)

但是，我收到一条错误消息:

<块引用>

ValueError: ('All cell must have a state_size attribute. received cells:', [])

我该如何纠正?

解决方案

感谢 @qlzh727.在这里，我引用回复:

StackedRNNCells 或 StackedRNNCells 仅适用于 Cell，而不适用于层.RNN中cell和layer的区别在于cell只会处理整个序列中的一个时间步，而layer会处理整个序列.您可以将 RNN 层视为:

for t in Whole_time_steps:output_t, state_t = cell(input_t, state_t-1)

如果你想在 1.x 中将 2 个 LSTM 层与 cudnn 堆叠在一起，你可以这样做:

l1 = tf.layers.CuDNNLSTM(128, return_sequence=True)l2 = tf.layers.CuDNNLSTM(128)l1_output = l1(输入)l2_输出= l2(l1_输出)

在 tf 2.x 中，我们将 cudnn 和普通实现统一到一起，您可以使用 tf.layers.LSTM(128, return_sequence=True) 更改上面的示例，如果可用，它将使用 cudnn impl.

I want to implement an RNN using Tensorflow1.13 on GPU. Following the official recommendation, I write the following code to get a stack of RNN cells

lstm = [tk.layers.CuDNNLSTM(128) for _ in range(2)]
cells = tk.layers.StackedRNNCells(lstm)

However, I receive an error message:

ValueError: ('All cells must have a state_size attribute. received cells:', [< tensorflow.python.keras.layers.cudnn_recurrent.CuDNNLSTM object at 0x13aa1c940>])

How can I correct it?

解决方案

Thanks @qlzh727. Here, I quote the response:

Either StackedRNNCells or StackedRNNCells only works with Cell, not layer. The difference between the cell and layer in RNN is that cell will only process one time step within the whole sequence, whereas the layer will process the whole sequence. You can treat RNN layer as:

for t in whole_time_steps:
  output_t, state_t = cell(input_t, state_t-1)

If you want to stack 2 LSTM layers to together with cudnn in 1.x, you can do:

l1 = tf.layers.CuDNNLSTM(128, return_sequence=True)
l2 = tf.layers.CuDNNLSTM(128)
l1_output = l1(input)
l2_oupput = l2(l1_output) 

In tf 2.x, we unify the cudnn and normal implementation together, you can just change the example above with tf.layers.LSTM(128, return_sequence=True), which will use the cudnn impl if available.

这篇关于如何在 Tensorflow 中实现堆叠的 RNN?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！