不应该model.trainable=模型下的假冻结权重? [英] shouldn't model.trainable=False freeze weights under the model?

问题描述

我正在尝试冻结免费训练的 VGG16 层(下面的conv_base")并在它们之上添加新层以进行特征提取.我希望从 'conv_base' before(ret1)/after(ret2) 模型拟合得到相同的预测结果，但事实并非如此.这是检查体重冻结的错误方法吗?

I am trying to freeze the free trained VGG16's layers ('conv_base' below) and add new layers on top of them for feature extracting. I expect to get same prediction results from 'conv_base' before(ret1) / after(ret2) fit of model but it is not. Is this wrong way to check weight freezing?

conv_base  = applications.VGG16(weights='imagenet', include_top=False, input_shape=[150, 150, 3]) 
conv_base.trainable = False

模型拟合前的结果

ret1 = conv_base.predict(np.ones([1, 150, 150, 3]))

在 VGG16 上添加层并编译模型

model = models.Sequential()
model .add(conv_base)
model .add(layers.Flatten())
model .add(layers.Dense(10, activation='relu'))
model .add(layers.Dense(1, activation='sigmoid'))
m.compile('rmsprop', 'binary_crossentropy', ['accuracy'])

拟合模型

m.fit_generator(train_generator, 100, validation_data=validation_generator, validation_steps=50)

模型拟合后的结果

ret2 = conv_base.predict(np.ones([1, 150, 150, 3]))

希望这是真的，但事实并非如此.

np.equal(ret1, ret2)

推荐答案

这是一个有趣的案例.为什么会发生这样的事情是由以下原因造成的:

This is an interesting case. Why something like this happen is caused by the following thing:

你不能在编译后冻结整个模型，如果没有编译它也不会被冻结

如果你设置了一个标志 model.trainable=False 然后在编译 keras 时将所有层设置为不可训练.如果您在编译后设置此标志 - 那么它根本不会影响您的模型.相同 - 如果您在编译之前设置此标志，然后您将重用模型的一部分来编译另一个模型 - 它不会影响您重用的层.所以 model.trainable=False 仅在您按以下顺序应用时才有效:

If you set a flag model.trainable=False then while compiling keras sets all layers to be not trainable. If you set this flag after compilation - then it will not affect your model at all. The same - if you set this flag before compiling and then you'll reuse a part of a model for compiling another one - it will not affect your reused layers. So model.trainable=False works only when you'll apply it in a following order:

# model definition
model.trainable = False
model.compile()

在任何其他情况下，它都不会按预期工作.

In any other scenario it wouldn't work as expected.

这篇关于不应该model.trainable=模型下的假冻结权重?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！