如何在Keras中为每个时间步应用不同的密集层 [英] How to apply a different dense layer for each timestep in Keras

问题描述

我知道应用TimeDistributed(Dense)在所有时间步长上都应用相同的密集层，但是我想知道如何为每个时间步长应用不同的密集层.时间步数不变.

I know that applying a TimeDistributed(Dense) applies the same dense layer over all the timesteps but I wanted to know how to apply different dense layers for each timestep. The number of timesteps is not variable.

PS:我看过以下链接，似乎找不到答案

P.S.: I have seen the following link and can't seem to find an answer

推荐答案

您可以使用LocallyConnected层.

You can use a LocallyConnected layer.

LocallyConnected层表示为连接到每个kernel_size time_steps(在此情况下为1)的密集层.

The LocallyConnected layer words as a Dense layer connected to each of kernel_size time_steps (1 in this case).

from tensorflow import keras
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model

sequence_length = 10
n_features = 4

def make_model():
  inp = Input((sequence_length, n_features))
  h1 = LocallyConnected1D(8, 1, 1)(inp)
  out = Flatten()(h1)
  model = Model(inp, out)
  model.compile('adam', 'mse')
  return model

model = make_model()
model.summary()

每个摘要中，LocallyConnected层使用的变量数为 (output_dims * (input_dims + bias)) * time_steps或(8 *(4 + 1))* 10 = 400.

Per summary the number of variables used by the LocallyConnected layer is (output_dims * (input_dims + bias)) * time_steps or (8 * (4 + 1)) * 10 = 400.

用另一种方式表达:上面的本地连接层表现为10个不同的Dense层，每个层都连接到其时间步长(因为我们选择kernel_size为1).这些包含50个变量的块中的每个块都是形状的权重矩阵(input_dims，output_dims)加上大小的偏差矢量(output_dims).

Wording it another way: the locally connected layer above behaves as 10 different Dense layers each connected to its time step (because we choose kernel_size as 1). Each of these blocks of 50 variables, is a weights matrix of shape (input_dims, output_dims) plus a bias vector of size (output_dims).

还要注意，给定input_shape为(sequence_len，n_features)，Dense(output_dims)和Conv1D(output_dims, 1, 1)是等效的.

Also note that given an input_shape of (sequence_len, n_features), Dense(output_dims) and Conv1D(output_dims, 1, 1) are equivalent.

即此模型:

def make_model():
  inp = Input((sequence_length, n_features))
  h1 = Conv1D(8, 1, 1)(inp)
  out = Flatten()(h1)
  model = Model(inp, out)

和此模型:

def make_model():
  inp = Input((sequence_length, n_features))
  h1 = Dense(8)(inp)
  out = Flatten()(h1)
  model = Model(inp, out)

都一样.

这篇关于如何在Keras中为每个时间步应用不同的密集层的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！