如何计算 LSTM 网络的参数数量? [英] How to calculate the number of parameters of an LSTM network?

问题描述

有没有办法计算 LSTM 网络中的参数总数.

Is there a way to calculate the total number of parameters in a LSTM network.

我找到了一个例子，但我不确定这是或者如果我理解正确的话.

I have found a example but I'm unsure of how correct this is or If I have understood it correctly.

例如考虑以下示例:-

from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation
from keras.layers import Embedding
from keras.layers import LSTM
model = Sequential()
model.add(LSTM(256, input_dim=4096, input_length=16))
model.summary()

输出

____________________________________________________________________________________________________
Layer (type)                       Output Shape        Param #     Connected to                     
====================================================================================================
lstm_1 (LSTM)                      (None, 256)         4457472     lstm_input_1[0][0]               
====================================================================================================
Total params: 4457472
____________________________________________________________________________________________________

根据我的理解 n 是输入向量长度.m 是时间步数.在这个例子中，他们认为隐藏层的数量为 1.

As per My understanding n is the input vector lenght. And m is the number of time steps. and in this example they consider the number of hidden layers to be 1.

因此根据帖子中的公式.4(nm+n^2) 在我的例子中 m=16;n=4096;num_of_units=256

Hence according to the formula in the post. 4(nm+n^2) in my example m=16;n=4096;num_of_units=256

4*((4096*16)+(4096*4096))*256 = 17246978048

为什么会有这么大的差别?是我误解了这个例子还是公式有误?

Why is there such a difference? Did I misunderstand the example or was the formula wrong ?

推荐答案

No - Keras 中一个 LSTM 层的参数个数等于:

No - the number of parameters of a LSTM layer in Keras equals to:

params = 4 * ((size_of_input + 1) * size_of_output + size_of_output^2)

额外的 1 来自偏见条款.所以 n 是输入的大小(由偏置项增加)，m 是 LSTM 层的输出大小.

Additional 1 comes from bias terms. So n is size of input (increased by the bias term) and m is size of output of a LSTM layer.

最后:

4 * (4097 * 256 + 256^2) = 4457472

这篇关于如何计算 LSTM 网络的参数数量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！