如何在Pytorch中的每个存储块中实现具有多个单元的LSTM层? [英] How to implement LSTM layer with multiple cells per memory block in Pytorch?

问题描述

我打算在Pytorch中实现LSTM，每层具有多个存储单元块-或多个LSTM单元，一个LSTM单元是存储块及其门的集合-每层，但似乎基类torch.nn.LSTM启用了仅用于实现每层一个LSTM单元的多层LSTM:

I intend to implement an LSTM in Pytorch with multiple memory cell blocks - or multiple LSTM units, an LSTM unit being the set of a memory block and its gates - per layer, but it seems that the base class torch.nn.LSTM enables only to implement a multi-layer LSTM with one LSTM unit per layer:

lstm = torch.nn.LSTM(input_size, hidden_size, num_layers)

其中(根据Pytorch的文档):

where (from the Pytorch's documentation):

• input_size是网络的输入维度，
• hidden_size是每一层的隐藏状态维度(即每一层的维度)，
• num_layer是网络的层数

• input_size is the input dimension of the network,
• hidden_size is the hidden state dimension for every layer (i.e. the dimension of every layer),
• num_layer is the number of layers of the network

因此，从上方看，每个LSTM单元正好具有一个单元(因此，每个LSTM单元的单元状态是标量)，因为对于每一层，单元状态的维数对应于隐藏状态的维数(即).

Thereupon, from above, each LSTM unit has exactly one cell (the cell state for each LSTM unit is thus a scalar) because for each layer the dimension of the cell state corresponds to the dimension of the hidden state (i.e. hidden_size).

但是，在Hochreiter和Schmidhuber提出的原始LSTM模型中 [1997]，每个LSTM块/单元可以包含几个单元: LSTM网络[Hochreiter，1997]

However in the original LSTM model proposed by Hochreiter and Schmidhuber [1997], each LSTM block/unit can contains several cells: LSTM Network [Hochreiter, 1997]

有办法吗?

推荐答案

据我了解，两个LSTM单元照常连接.本文对图2的描述如下:

For my understanding, the two LSTM-Cells are connected as usual. The description of Figure 2 in this paper, says following:

...(请注意，通过将图1逆时针旋转90度，它将与图2的相应部分匹配)...

...(note that by rotating Figure 1 by 90 degrees anticlockwise, it will match with corresponding parts of Figure 2)...

图2中的图形可能很难解释，但它应该是LSTM，其中包含hidden_​​layer和state的输入.

The graph in Figure 2 is probably hard to interpret, but it should be a LSTM with the input of hidden_layer and the state.

这篇关于如何在Pytorch中的每个存储块中实现具有多个单元的LSTM层?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！