在Keras中构建CNN + LSTM，以解决回归问题.什么是合适的形状? [英] Building CNN + LSTM in Keras for a regression problem. What are proper shapes?

问题描述

我正在研究一个回归问题，在该问题中，我将一组频谱图输入到CNN + LSTM(喀拉拉邦的体系结构)中.我的数据的形状为(n_samples, width, height, n_channels).我有一个问题，如何将CNN正确连接到LSTM层.当卷积传递到LSTM时，需要以某种方式对数据进行整形.有几种想法，例如将TimeDistributed -wrapper与重塑结合使用，但我无法使其正常工作.

I am working on a regression problem where I feed a set of spectograms to CNN + LSTM - architecture in keras. My data is shaped as (n_samples, width, height, n_channels). The question I have how to properly connect the CNN to the LSTM layer. The data needs to be reshaped in some way when the convolution is passed to the LSTM. There are several ideas, such as use of TimeDistributed-wrapper in combination with reshaping but I could not manage to make it work. .

height = 256
width = 256
n_channels = 3
seq_length = 1 #?

---

我是从这个网络开始的:

---

I started out with this network:

i = Input(shape=(width, height, n_channels))
    conv1 = Conv2D(filters=32,
                   activation='relu',
                   kernel_size=(2, 2),
                   padding='same')(i)
    lstm1 = LSTM(units=128,
                 activation='tanh',
                 return_sequences=False)(conv1)
    o = Dense(1)(lstm1)

我得到的错误是:

ValueError: Input 0 of layer lstm is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 256, 256, 32]

---

我找到了 thread 中给出的信息的示例.它需要添加TimeDistributed-包装器.

---

I found a thread suggesting to reshape. Below is an example of how I applied the information given in thread here. It requires to add the TimeDistributed-Wrapper.

i = Input(shape=(seq_length, width, height, n_channels))
conv1 = TimeDistributed(Conv2D(filters=32,
               activation='relu',
               kernel_size=(2, 2),
               padding='same'))(i)
conv1 = Reshape((seq_length, height*width*n_channels))(conv1)
lstm1 = LSTM(units=128,
             activation='tanh',
             return_sequences=False)(conv1)
o = Dense(1)(lstm1)

结果是:

ValueError: Error when checking input: expected input_1 to have 5 dimensions, but got array with shape (5127, 256, 256, 3)

---

但是，在上述SO的示例中，网络是在视频序列上训练的，因此需要TimeDistributed(?).就我而言，我有一组来自信号的频谱图，而我不是在训练视频.因此，一个想法是将time_steps添加到1来克服这一问题. 在此处进行了类似的操作.输入层如下:

---

In the example from the SO above, however, the network is trained on video sequence and thus the need for TimeDistributed(?). In my case, I have a set of spectograms that originate from a signal and I am not training a video. So, an idea was to add the time_steps to 1 to overcome this. Something similar was done here. The input layer is then:

Input(shape=(seq_length, width, height, n_channels))

导致重塑动作出错.

ValueError: total size of new array must be unchanged

---

我希望在如何正确连接CNN + LSTM层方面提供一些帮助.谢谢！

---

I'd appreciate some help on how properly connect the CNN + LSTM layers. Thank you!

推荐答案

一种可能的解决方案是将LSTM输入设置为形状(num_pixels, cnn_features).在您的特定情况下，如果cnn具有32个过滤器，则LSTM将收到(256*256, 32)

One possible solution is setting the LSTM input to be of shape (num_pixels, cnn_features). In your particular case, having a cnn with 32 filters, the LSTM would receive (256*256, 32)

cnn_features = 32

inp = tf.keras.layers.Input(shape=(256, 256, 3))
x = tf.keras.layers.Conv2D(filters=cnn_features,
                   activation='relu',
                   kernel_size=(2, 2),
                   padding='same')(inp)
x = tf.keras.layers.Reshape((256*256, cnn_features))(x)
x = tf.keras.layers.LSTM(units=128,
        activation='tanh',
        return_sequences=False)(x)
out = tf.keras.layers.Dense(1)(x)

这篇关于在Keras中构建CNN + LSTM，以解决回归问题.什么是合适的形状?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！