在tf.nn.conv2d中使用权重初始值设定项 [英] Using weights initializer with tf.nn.conv2d

问题描述

使用 tf.layers.conv2d 时，设置初始化程序很容易，可以通过其参数来完成.但是，如果我使用 tf.nn.conv2d ，该怎么办?我使用此代码.这是否等同于在 tf.layers.conv2d 中设置 kernel_initializer 参数?尽管程序运行没有错误，但我不知道如何验证程序是否按预期进行.

When using tf.layers.conv2d, setting the initializer is easy, it can be done through its parameter. But what if I use tf.nn.conv2d? I use this code. Is this equivalent to setting the kernel_initializer parameter in tf.layers.conv2d? Although the program runs without errors, I don't know how to verify whether it does what it is expected do.

 with tf.name_scope('conv1_2') as scope:
        kernel = tf.get_variable(initializer=tf.contrib.layers.xavier_initializer(), 
                                 shape=[3, 3, 32, 32], name='weights')
        conv = tf.nn.conv2d(conv1_1, kernel, [1, 1, 1, 1], padding='SAME')
        biases = tf.Variable(tf.constant(0.0, shape=[32], dtype=tf.float32),
                             trainable=True, name='biases')
        out = tf.nn.bias_add(conv, biases)
        self.conv1_2 = tf.nn.relu(out, name=scope)
        self.parameters += [kernel, biases]

推荐答案

下面的操作相同(请参见

The operation underneath is the same (see here).

关于内核及其初始化，我瞥了一眼代码，看起来相同 ... layers.conv2d 调用了 tf.get_variable 在一天结束时.

As for the kernel and its initialization, I took a glimpse in the code and it looked the same... the layers.conv2d call a tf.get_variable at the end of the day.

但是我想凭经验看一下，所以这是一个测试代码，它使用每种方法( tf.layers.conv2d 和 tf.nn.conv2d )，评估初始化后的内核并进行比较.

But I wanted to see it empirically, so here is a test code that declares a conv2d using each method (tf.layers.conv2d and tf.nn.conv2d), evaluates the initialized kernels and compares them.

我已经任意设置了不应影响比较的内容，例如输入张量和步幅.

I've arbitrarily set the things that shouldn't interfere in the comparison, such as an input tensor and the strides.

import tensorflow as tf
import numpy as np


# the way you described in your question
def _nn(input_tensor, initializer, filters, size):
    kernel = tf.get_variable(
        initializer=initializer, 
        shape=[size, size, 32, filters],
        name='kernel')

    conv = tf.nn.conv2d(
        input=input_tensor,
        filter=kernel,
        strides=[1, 1, 1, 1],
        padding='SAME')

    return kernel

# the other way
def _layer(input_tensor, initializer, filters, size):
    tf.layers.conv2d(
        inputs=input_tensor,
        filters=filters,
        kernel_size=size,
        kernel_initializer=initializer)

    # 'conv2d/kernel:0' is the name of the generated kernel
    return tf.get_default_graph().get_tensor_by_name('conv2d/kernel:0')

def _get_kernel(method):
    # an isolated context for each conv2d
    graph = tf.Graph()
    sess = tf.Session(graph=graph)

    with graph.as_default(), sess.as_default():
        # important so that same randomness doesnt play a role
        tf.set_random_seed(42)

        # arbitrary input tensor with compatible shape
        input_tensor = tf.constant(1.0, shape=[1, 64, 64, 32])

        initializer = tf.contrib.layers.xavier_initializer()

        kernel = method(
            input_tensor=input_tensor,
            initializer=initializer,
            filters=32,
            size=3)

        sess.run(tf.global_variables_initializer())
        return sess.run(kernel)

if __name__ == '__main__':
    kernel_nn = _get_kernel(_nn)
    kernel_layer = _get_kernel(_layer)

    print('kernels are ', end='')
    # compares shape and values
    if np.array_equal(kernel_layer, kernel_nn):
        print('exactly the same')
    else:
        print('not the same!')

输出为... 内核完全相同.

文档，顺便说一句: tf.nn.conv2d 和 tf.layers.conv2d .

The docs, btw: tf.nn.conv2d and tf.layers.conv2d.

这篇关于在tf.nn.conv2d中使用权重初始值设定项的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！