TensorFlow:在部分加载预训练权重后使用 tf.global_variables_initializer() [英] TensorFlow: Using tf.global_variables_initializer() after partially loading pre-trained weights

问题描述

我构建了一个模型，该模型使用预训练的 VGG-16 作为基础网络，然后在此基础上添加了几层.

I built a model that uses a pre-trained VGG-16 as a base network and then adds a few layers on top of that.

在训练时，我的模型部分由属于预训练 VGG-16 的变量组成，即已经初始化的变量(整个模型正在从 SavedModel 协议缓冲区加载)，部分由来自我添加的图层.

At training time, my model consists partially of variables that belong to the pre-trained VGG-16, i.e. variables that are already initialized (the entire model is being loaded from a SavedModel protocol buffer), and partially of uninitialized variables from the layers I added.

在开始训练之前，我必须初始化我在预训练的 VGG-16 之上添加的层的变量.我通过运行 tf.global_variables_initializer() 来做到这一点.

Before I start the training, I have to initialize the variables of the layers I added on top of the pre-trained VGG-16. I do that by running tf.global_variables_initializer().

问题来了:

如果 tf.global_variables_initializer() 初始化所有全局变量，为什么它不使用初始化值覆盖预训练的权重?

If tf.global_variables_initializer() initializes all global variables, why does it not override the pre-trained weights with their initialization values?

推荐答案

事实上，它确实覆盖了预训练的权重:

In fact, it does override pre-trained weights:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import backend

session = tf.Session ()
backend.set_session (session)
with session.as_default ():
    net = keras.applications.InceptionV3 (input_shape = (3, 299, 299))
    print ('weights before: ', net.get_layer ('conv2d_1').get_weights ())

    session.run (tf.global_variables_initializer ())

    print ('weights after: ', net.get_layer ('conv2d_1').get_weights ())

输出:

weights before:  [array([[[[ 7.42468759e-02,  7.99097791e-02, -1.20985091e-01, ...,
      -2.42504068e-02, -8.17770883e-03, -7.11583123e-02],
     [ 5.46021610e-02,  1.13289319e-01,  3.99106629e-02, ...,
       2.37093717e-02, -6.01868033e-02, -1.08440826e-02],
     [ 1.00897752e-01,  1.96732566e-01,  8.93706456e-02, ...,
       1.19806841e-01, -9.84039158e-02, -7.20797330e-02],

weights after:  [array([[[[-7.92664364e-02, -5.87778017e-02, -9.76844281e-02, ...,
       9.62431133e-02,  7.16331154e-02, -3.22120935e-02],
     [-6.17715716e-03, -6.95393384e-02,  9.68316942e-02, ...,
       5.01702428e-02,  8.89533758e-02,  9.80506092e-02],
     [-1.89049393e-02, -4.31398787e-02, -9.45695043e-02, ...,
       7.52686858e-02,  7.27956891e-02, -3.26380134e-03],

为了克服这个问题，将自定义变量放在一个单独的集合中:

To overcome this, put custom variables in a separate collection:

my_variables_collection = tf.get_collection ('my_variables')

t = tf.Variable (0.0, name='adam_t', trainable=False)
tf.add_to_collection ('my_variables', t)

m = tf.Variable (backend.zeros ((3, 299, 299)), name='adam_m', trainable=False)
tf.add_to_collection ('my_variables', m)

# only initialize variables from my list:
session.run (tf.variables_initializer (my_variables_collection))

这篇关于TensorFlow:在部分加载预训练权重后使用 tf.global_variables_initializer()的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！