Tensorflow中RNN的LSTMStateTuple vs cell.zero_state() [英] LSTMStateTuple vs cell.zero_state() for RNN in Tensorflow

问题描述

我对在Tensorflow中为RNN使用初始状态张量的正确方法感到困惑.使用 LSTMStateTuple 或 cell.zero_state .

两个都一样吗?如果是这样，为什么有两种方法?

在一个示例中，他们使用tf.nn.rnn_cell.LSTMStateTuple设置初始状态，而在另一个示例中，他们使用cell.zero_state().

为什么有两种方法?什么时候更喜欢一个或另一个?设置state_is_tuple时只能使用LSTMStateTuple吗?如果是这样，cell.zero_state()不再有效吗?

解决方案

两者是不同的东西. state_is_tuple用于LSTM单元，因为LSTM单元的状态是元组. cell.zero_state是所有RNN单元的状态的初始化程序.

您通常会更喜欢cell.zero_state函数，因为它将根据state_is_tuple是否为true来初始化所需的状态类.

请参见此 GitHub问题，您可以在其中看到推荐的cell.zero_state-使用单元格对象上的zero_state函数".

您可能想要cell.zero_state的另一个原因是，它与单元格的类型(LSTM，GRU，RNN)无关，因此您可以执行以下操作:

if type == 'GRU':
   cell = BasicGRUCell
else:
   cell = BasicLSTMCell(state_is_tuple=True)

init_state = cell.zero_state(batch_size)

将初始状态设置为OK.

LSTMStateTuple仅适用于状态为元组的单元格.

何时使用LSTMStateTuple?

在使用自定义值(由培训师通过)初始化状态时，将要使用LSTMStateTuple. cell.zero_state()将返回所有等于0.0的状态.

如果要在批次之间保持状态，则必须在每个批次之后获取状态，然后将其添加到下一个批次的feed_dict中.

有关为什么LSTM状态是元组的说明，请参见此

I am confused on what is the correct way to use the initial state tensor in Tensorflow for RNNs. There is almost a 50/50 split between turtorials that either use LSTMStateTuple or cell.zero_state.

Are the two the same? If so, why are there two ways?

In one example they use tf.nn.rnn_cell.LSTMStateTuple to set the initial state and in the other they use cell.zero_state().

Why are there two methods? When to prefer the one or the other? Can you only use LSTMStateTuple when you set state_is_tuple? If so, does cell.zero_state() no longer work?

解决方案

The two are different things. state_is_tuple is used on LSTM cells because the state of LSTM cells is a tuple. cell.zero_state is the initializer of the state for all RNN cells.

You will generally prefer cell.zero_state function as it will initialize the required state class depending on whether state_is_tuple is true or not.

See this GitHub issue where you can see the cell.zero_state recommended - "use the zero_state function on the cell object".

Another reason why you may want cell.zero_state is because it is agnostic of the type of the cell (LSTM, GRU, RNN) and you can do something like this:

if type == 'GRU':
   cell = BasicGRUCell
else:
   cell = BasicLSTMCell(state_is_tuple=True)

init_state = cell.zero_state(batch_size)

with the initial state being set up OK.

LSTMStateTuple will work only on cells that have the state as a tuple.

When to use LSTMStateTuple?

You'll want to use LSTMStateTuple when you're initializing your state with custom values (passed by the trainer). cell.zero_state() will return the state with all the values equal to 0.0.

If you want to keep state between batches than you'll have to get it after each batch and add it to your feed_dict the next batch.

See this for an explanation on why LSTM state is a tuple.

这篇关于Tensorflow中RNN的LSTMStateTuple vs cell.zero_state()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！