TensorFlow 2.0 如何从 tf.keras.layers 层获取可训练变量，如 Conv2D 或 Dense [英] TensorFlow 2.0 How to get trainable variables from tf.keras.layers layers, like Conv2D or Dense

问题描述

我一直试图从我的层中获取可训练的变量，但无法找到使其工作的方法.所以这是我尝试过的:

I have been trying to get the trainable variables from my layers and can't figure out a way to make it work. So here is what I have tried:

我曾尝试直接访问 Dense 或 Conv2D 对象的内核和偏差属性，但无济于事.我得到的结果类型是密集对象没有属性‘内核’".

I have tried accessing the kernel and bias attribute of the Dense or Conv2D object directly, but to no avail. The type of result that I get is "Dense object has no attribute 'kernel'".

trainable_variables.append(conv_layer.kernel)
trainable_variables.append(conv_layer.bias)

同样，我尝试通过以下方式使用属性trainable_variables":

Similarly, I have tried using the attribute "trainable_variables" in the following way:

trainable_variables.extend(conv_layer.trainable_variables)

据我所知，这应该返回两个变量的列表，权重和偏差变量.但是，我得到的是一个空列表.

From what I know this is supposed to return a list of two variables, the weight and the bias variables. However, what I get is an empty list.

知道如何从 TensorFlow 2.0 中的标签获取变量吗?我希望以后能够以类似于以下方式将这些变量提供给优化器:

Any idea of how to get the variables from labels in TensorFlow 2.0? I want to be able to later feed those variables to an optimizer, in a way similar to the following:

gradients = tape.gradient(loss, trainable_variables)
optimizer.apply_gradients(zip(gradients, trainable_variables))

这是我当前代码的一部分，用作示例并帮助回答问题(希望它是可读的)

Here is part of my current code to serve as an example and help answering the question (Hope it is readable)

from tensorflow.keras.layers import Dense, Conv2D, Conv2DTranspose, Reshape, Flatten

... 

class Network:
    def __init__(self, params):
        weights_initializer = tf.initializers.GlorotUniform(seed=params["seed"])
        bias_initializer = tf.initializers.Constant(0.0)

        self.trainable_variables = []

        self.conv_layers = []
        self.conv_activations = []
        self.create_conv_layers(params, weights_initializer, bias_initializer)

        self.flatten_layer = Flatten()


        self.dense_layers = []
        self.dense_activations = []
        self.create_dense_layers(params, weights_initializer, bias_initializer)

        self.output_layer = Dense(1, kernel_initializer=weights_initializer, bias_initializer=bias_initializer)
        self.trainable_variables.append(self.output_layer.kernel)
        self.trainable_variables.append(self.output_layer.bias)

    def create_conv_layers(self, params, weight_init, bias_init):
        nconv = len(params['stride'])
        for i in range(nconv):
            conv_layer = Conv2D(filters=params["nfilter"][i],
                                kernel_size=params["shape"][i], kernel_initializer=weight_init,
                                kernel_regularizer=spectral_norm,
                                use_bias=True, bias_initializer=bias_init,
                                strides=params["stride"][i],
                                padding="same", )
            self.conv_layers.append(conv_layer)
            self.trainable_variables.append(conv_layer.kernel)
            self.trainable_variables.append(conv_layer.bias)
            self.conv_activations.append(params["activation"])

    def create_conv_layers(self, params, weight_init, bias_init):
        nconv = len(params['stride'])
        for i in range(nconv):
            conv_layer = Conv2D(filters=params["nfilter"][i],
                                kernel_size=params["shape"][i], kernel_initializer=weight_init,
                                kernel_regularizer=spectral_norm,
                                use_bias=True, bias_initializer=bias_init,
                                strides=params["stride"][i],
                                padding="same", )
            self.conv_layers.append(conv_layer)
            self.trainable_variables.append(conv_layer.kernel)
            self.trainable_variables.append(conv_layer.bias)
            self.conv_activations.append(params["activation"])

如您所见，我正在尝试将所有可训练变量收集到一个名为 trainable_variables 的列表属性中.然而，正如我所提到的，这段代码不起作用，因为我在尝试获取这些层对象的内核和偏差属性时出错.

As you can see I am trying to gather all my trainable variables into a list attribute called trainable_variables. However as I mentioned this code does not work because I get an error for trying to acquire the kernel and bias attributes of those layer objects.

推荐答案

好的，我想我找到了问题.

Ok, so I think I found the problem.

在我使用给定的图层对象之前，可训练变量不可用.运行前向传递后，我可以检索 tf.keras.layers.Layer 对象的属性，如 trainable_variables 和权重.

The trainable variables were not available until I used the given layer object. After I run my forward pass I could retrieve attributes of the tf.keras.layers.Layer object like trainable_variables and weights.

然而，在我向前传球之前，我收到了一个空名单.为了让事情更清楚一点:

However, before my forward pass I received an empty list. To make things a little bit more clear:

with tf.GradientTape() as tape:
    print(dense_layers[0].trainable_variables)
    self.forward_pass(X)
    self.compute_loss()
    print(dense_layers[0].trainable_variables)

在上面的代码中，属性trainable_variables在执行self.forward_pass之前是一个空列表.然而，在它之后，我可以检索内核和偏置 numpy 数组.

On the code above, the attribute trainable_variables is an empty list before executing self.forward_pass. However, right after it, I could retrieve the kernel and bias numpy arrays.

这篇关于TensorFlow 2.0 如何从 tf.keras.layers 层获取可训练变量，如 Conv2D 或 Dense的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！