CNN的培训和测试准确性没有增加，其次是RNN以进行签名验证 [英] Training and Testing accuracy not increasing for a CNN followed by a RNN for signature verification

问题描述

我目前正在从事在线签名验证.数据集具有(x，7)的可变形状，其中x是一个人用来签名的点数.我有以下模型:

I'm currently working on online signature verification. The dataset has a variable shape of (x, 7) where x is the number of points a person used to sign their signature. I have the following model:

    model = Sequential()
    #CNN
    model.add(Conv1D(filters=64, kernel_size=3, activation='sigmoid', input_shape=(None, 7)))
    model.add(MaxPooling1D(pool_size=3))
    model.add(Conv1D(filters=64, kernel_size=2, activation='sigmoid'))

    #RNN
    model.add(Masking(mask_value=0.0))
    model.add(LSTM(8))
    model.add(Dense(2, activation='softmax'))

    opt = Adam(lr=0.0001)
    model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
    model.summary()

    print(model.fit(x_train, y_train, epochs=100, verbose=2, batch_size=50))

    score, accuracy = model.evaluate(x_test,y_test, verbose=2)
    print(score, accuracy)

我知道这可能不是最好的模型，但这是我第一次构建神经网络.我必须使用CNN和RNN，这是我的荣誉项目所需的.目前，我达到了最高的训练准确度0.5142和测试准确度0.54.我尝试增加纪元数，更改激活功能，添加更多层，四处移动各层，更改学习率并更改优化程序.

I know it may not be the best model but this is the first time I'm building a neural network. I have to use a CNN and RNN as it is required for my honours project. At the moment, I achieve 0.5142 as the highest training accuracy and 0.54 testing accuracy. I have tried increasing the number of epochs, changing the activation function, add more layers, moving the layers around, changing the learning rate and changing the optimizer.

请分享一些有关更改模型或数据集的建议.任何帮助深表感谢.

Please share some advice on changing my model or dataset. Any help is much appreciated.

推荐答案

对于CNN-RNN，可以尝试一些有前途的事情:

For CNN-RNN, some promising things to try:

• Conv1D层:activation='relu'，kernel_initializer='he_normal'
• LSTM层:activation='tanh'和recurrent_dropout=.1, .2, .3
• Optimizer :Nadam，lr=2e-4(Nadam可能明显胜过RNN的所有其他优化器)
• 批处理大小:降低它.除非您总共有200个以上的批次，否则请设置batch_size=32；否则，请使用batch_size=32.较低的批处理大小更好地利用了优化程序的随机机制，并可以提高泛化性
• 退出:在第二个Conv1D之后，速率为.1, .2-在第一个Conv1D之后，速率为.25, .3，但仅 如果您使用SqueezeExcite(请参见下文)，则MaxPooling不能正常工作
• SqueezeExcite :可以增强各种任务中所有CNN的性能；您可以在下面使用Keras实施
• BatchNormalization :虽然模型不大，但模型仍然很深，可能会在第二个Conv1D
之后立即从一个BN层中受益
• L2权重衰减:首先在 Conv1D上进行，以防止其记住输入内容；尝试1e-5, 1e-4，例如kernel_regularizer=l2(1e-4) # from keras.regularizers import l2
• 预处理:确保所有数据均已规范化(或按时间序列进行标准化)，并且在每个时期都对批次进行了混洗

• Conv1D layers: activation='relu', kernel_initializer='he_normal'
• LSTM layer: activation='tanh', and recurrent_dropout=.1, .2, .3
• Optimizer: Nadam, lr=2e-4 (Nadam may significantly outperform all other optimizers for RNNs)
• batch_size: lower it. Unless you have 200+ batches in total, set batch_size=32; lower batch size better exploits the Stochastic mechanism of the optimizer and can improve generalization
• Dropout: right after second Conv1D, with a rate .1, .2 - or, after first Conv1D, with a rate .25, .3, but only if you use SqueezeExcite (see below), else MaxPooling won't work as well
• SqueezeExcite: shown to enhance all CNN performance across a large variety of tasks; Keras implementation you can use below
• BatchNormalization: while your model isn't large, it's still deep, and may benefit from one BN layer right after second Conv1D
• L2 weight decay: on first Conv1D, to prevent it from memorizing the input; try 1e-5, 1e-4, e.g. kernel_regularizer=l2(1e-4) # from keras.regularizers import l2
• Preprocessing: make sure all data is normalized (or standardized if time-series), and batches are shuffled each epoch

def SqueezeExcite(_input):
    filters = _input._keras_shape[-1]

    se = GlobalAveragePooling1D()(_input)
    se = Reshape((1, filters))(se)
    se = Dense(filters//16,activation='relu',   
               kernel_initializer='he_normal', use_bias=False)(se)
    se = Dense(filters,    activation='sigmoid',
               kernel_initializer='he_normal', use_bias=False)(se)

    return multiply([_input, se])

# Example usage
x = Conv1D(filters=64, kernel_size=4, activation='relu', kernel_initializer='he_normal')(x)
x = SqueezeExcite(x) # place after EACH Conv1D

这篇关于CNN的培训和测试准确性没有增加，其次是RNN以进行签名验证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！