转移学习-Val_loss异常行为 [英] Transfer Learning - Val_loss strange behaviour
问题描述
我正在尝试从phyton中的keras.application对MobileNetV2使用转移学习。
我的图像属于4类，第一，第二，第三和最后一类的数量分别为8000、7000、8000和8000张图像。我的图像是灰度的，并从1024x1024调整为128x128。
I am trying to use transfer-learning on MobileNetV2 from keras.application in phyton. My images belongs to 4 classes with an amount of 8000, 7000, 8000 and 8000 images in the first, second, third and last class. My images are gray-scaled and resized from 1024x1024 to 128x128.
我从MobileNetV2中删除了分类密集层，并添加了自己的密集层：
I removed the classification dense layers from MobileNetV2 and added my own dense layers:
global_average_pooling2d_1 (Glo Shape = (None, 1280) 0 Parameters
______________________________________________________________________________
dense_1 (Dense) Shape=(None, 4) 5124 Parameters
______________________________________________________________________________
dropout_1 (Dropout) Shape=(None, 4) 0 Parameters
________________________________________________________________
dense_2 (Dense) Shape=(None, 4) 20 Parameters
__________________________________________________________________________
dense_3 (Dense) Shape=(None, 4) 20 Parameters
Total params: 2,263,148
Trainable params: 5,164
Non-trainable params: 2,257,984
如您所见，我添加了2个密集层，其中包含dropout作为正则化器。
Furhtermore，我使用了以下
As you can see I added 2 dense layers with dropout as regularizer. Furhtermore, I used the following
opt = optimizers.SGD(lr=0.001, decay=4e-5, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
batch_size = 32
我的训练结果很奇怪...：
My results on training are very weird... :
时代
1亏损：1.3378-acc：0.3028-val_loss：1.4629-val_acc：0.2702
1 loss: 1.3378 - acc: 0.3028 - val_loss: 1.4629 - val_acc: 0.2702
2亏损： 1.2807-acc：0.3351-val_loss：1.3297-val_acc：0.3208
2 loss: 1.2807 - acc: 0.3351 - val_loss: 1.3297 - val_acc: 0.3208
3损失：1.2641-acc：0.3486-val_loss：1.4428-val_acc：0.3707
3 loss: 1.2641 - acc: 0.3486 - val_loss: 1.4428 - val_acc: 0.3707
4亏损：1.2178-acc：0.3916-val_loss：1.4231-val_acc：0.3758
4 loss: 1.2178 - acc: 0.3916 - val_loss: 1.4231 - val_acc: 0.3758
5亏损：1.2100-acc：0.3909-val_loss ：1.4009-val_acc：0.3625
5 loss: 1.2100 - acc: 0.3909 - val_loss: 1.4009 - val_acc: 0.3625
6损失：1.1979-acc：0.3976-val_loss：1.5025-val_acc：0.3116
6 loss: 1.1979 - acc: 0.3976 - val_loss: 1.5025 - val_acc: 0.3116
7亏损：1.1943-acc：0.3988-val_loss：1.4510-val_acc：0.2872
7 loss: 1.1943 - acc: 0.3988 - val_loss: 1.4510 - val_acc: 0.2872
8亏损：1.1926-acc：0 .3965-val_loss：1.5162-val_acc：0.3072
8 loss: 1.1926 - acc: 0.3965 - val_loss: 1.5162 - val_acc: 0.3072
9损失：1.1888-acc：0.4004-val_loss：1.5659-val_acc：0.3304
9 loss: 1.1888 - acc: 0.4004 - val_loss: 1.5659 - val_acc: 0.3304
10亏损：1.1906-acc：0.3969-val_loss：1.5655-val_acc：0.3260
10 loss: 1.1906 - acc: 0.3969 - val_loss: 1.5655 - val_acc: 0.3260
11亏损：1.1864-acc：0.3999-val_loss：1.6286- val_acc：0.2967
11 loss: 1.1864 - acc: 0.3999 - val_loss: 1.6286 - val_acc: 0.2967
（...）
总而言之，训练的损失不再减少，而是仍然很高。该模型也过拟合。
您可能会问为什么我只添加了2个密集的层，每个层有4个神经元。在开始的时候，我尝试了不同的配置（例如128个神经元和64个神经元，以及不同的调节器），然后过拟合是一个巨大的问题，即训练的准确性几乎为1，而测试的损失仍远离0。
Summarizing, the loss of training does not decrease anymore and is still very high. The model also overfits. You may ask why I added only 2 dense layers with 4 neurons in each. In the beginning I tried different configurations (e.g. 128 neurons and 64 neurons and also different regulaziers), then overfitting was a huge problem, i.e. accuracy on training was almost 1 and loss on test was still far away from 0.
我有点困惑，因为这里有很大的错误。
I am a little bit confused what is going on, since something tremendously is wrong here.
微调的尝试：
分类部分的密集层中神经元的数量从1024到4不等。
不同的学习率（0.01，0.001，0.0001）
不同的批量大小（16,32，64）
具有0.001、0.0001的不同调节器L1
Fine-tuning attempts: Different numbers of neurons in the dense layers in the classification part varying from 1024 to 4. Different learning rates (0.01, 0.001, 0.0001) Different batch sizes (16,32, 64) Different regulaziers L1 with 0.001, 0.0001
结果：
总是过度拟合
Results: Always huge overfitting
base_model = MobileNetV2(input_shape=(128, 128, 3), weights='imagenet', include_top=False)
# define classificator
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(4, activation='relu')(x)
x = Dropout(0.8)(x)
x = Dense(4, activation='relu')(x)
preds = Dense(4, activation='softmax')(x) #final layer with softmax activation
model = Model(inputs=base_model.input, outputs=preds)
for layer in model.layers[:-4]:
layer.trainable = False
opt = optimizers.SGD(lr=0.001, decay=4e-5, momentum=0.9)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
batch_size = 32
EPOCHS = int(trainY.size/batch_size)
H = model.fit(trainX, trainY, validation_data=(testX, testY), epochs=EPOCHS, batch_size=batch_size)
结果应该是没有过度拟合且val_loss接近0。我从一些研究相似图像集的论文中知道。
Result should be that there is no overfitting and val_loss close to 0. I know that from some paper working on similiar image sets.
更新：
以下是一些val_loss，train_loss和准确性的图片：
2个密集层，具有16和8个神经元，lr = 0.001，衰减1e-6，batchsize = 25
推荐答案
可能是辍学率很高的结果。您没有显示数据生成器，所以我无法确定那里是否存在问题，但是我怀疑您需要使用
Could be the result of your dropout rate being to high. You do not show your data generators so I can't tell if there is an issue there but I suspect that you need to compile using
loss='sparse_categorical_crossentropy'
这篇关于转移学习-Val_loss异常行为的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
