keras何时重置LSTM状态? [英] When does keras reset an LSTM state?

问题描述

我阅读了各种各样的文字，似乎没有一个回答这个非常基本的问题.总是模棱两可:

I read all sorts of texts about it, and none seem to answer this very basic question. It's always ambiguous:

在stateful = False LSTM层中，keras是否在以下情况下重置状态:

In a stateful = False LSTM layer, does keras reset states after:

• 每个序列；或
• 每批?

假设我的X_train的形状为(1000,20,1)，表示1000个序列的20个步骤的单个值.如果我这样做:

Suppose I have X_train shaped as (1000,20,1), meaning 1000 sequences of 20 steps of a single value. If I make:

model.fit(X_train, y_train, batch_size=200, nb_epoch=15)

是否会为每个序列重置状态(重置状态为1000次)?
还是会为每个批次重置状态(重置状态为5次)?

Will it reset states for every single sequence (resets states 1000 times)?
Or will it reset states for every batch (resets states 5 times)?

推荐答案

通过一些测试，我得出了以下结论，该结论是根据文档以及Nassim的回答得出的:

Cheking with some tests, I got to the following conclusion, which is according to the documentation and to Nassim's answer:

首先，层中没有单一状态，但是批次中每个样品都有一个状态.在这样的层中有batch_size个并行状态.

First, there isn't a single state in a layer, but one state per sample in the batch. There are batch_size parallel states in such a layer.

在stateful=False情况下，每个批处理后所有状态都将一起重置.

In a stateful=False case, all the states are resetted together after each batch.

• 具有10 sequences的批处理将创建10 states，并且在处理完所有10个状态后会自动将其重置.

• A batch with 10 sequences would create 10 states, and all 10 states are resetted automatically after it's processed.

下一个具有10 sequences的批次将创建10 new states，该批次也将在处理该批次后重置

The next batch with 10 sequences will create 10 new states, which will also be resetted after this batch is processed

如果所有这些序列都具有length (timesteps) = 7，则这两个批次的实际结果是:

If all those sequences have length (timesteps) = 7, the practical result of these two batches is:

20个单独的序列，每个序列的长度为7

20 individual sequences, each with length 7

这些序列均不相关.但当然:权重(不是状态)对于该层而言是唯一的，并且将表示该层从所有序列中学到的知识.

None of the sequences are related. But of course: the weights (not the states) will be unique for the layer, and will represent what the layer has learned from all the sequences.

• 一种状态是:我现在在序列中的什么位置?现在是哪个时间步骤?自从开始到现在，这个特定序列的行为如何?
• 权重是:到目前为止，我对我所看到的所有序列的一般行为有什么了解?

在这种情况下，并行状态的数量也相同，但是根本不会被重置.

In this case, there is also the same number of parallel states, but they will simply not be resetted at all.

• 具有10 sequences的批处理将创建10 states，该批处理将在批处理结束时保持不变.

• A batch with 10 sequences will create 10 states that will remain as they are at the end of the batch.

具有10 sequences的下一批(必须是10，因为第一批是10)将重复使用与之前创建的相同的10 states.

The next batch with 10 sequences (it's required to be 10, since the first was 10) will reuse the same 10 states that were created before.

实际结果是:第二批中的10个序列正好接续了第一批中的10个序列，就好像根本没有中断一样.

The practical result is: the 10 sequences in the second batch are just continuing the 10 sequences of the first batch, as if there had been no interruption at all.

如果每个序列都具有length (timesteps) = 7，则实际含义是:

If each sequence has length (timesteps) = 7, then the actual meaning is:

10个单独的序列，每个序列的长度为14

10 individual sequences, each with length 14

当您看到达到序列的总长度时，请调用model.reset_states()，这意味着您将不再继续之前的序列，现在开始输入新序列.

When you see that you reached the total length of the sequences, then you call model.reset_states(), meaning you will not continue the previous sequences anymore, now you will start feeding new sequences.

这篇关于keras何时重置LSTM状态?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！