variable_scope 和 name_scope 之间的实际区别是什么? [英] What are the practical differences between the variable_scope and name_scope?

问题描述

虽然我已经浏览了关于同一个问题的页面:variable_scope 和 name_scope 之间有什么区别? 和 variable_ops_scope 和 variable_scope 有什么区别?.

Although I have gone through the pages regarding the same question: What is the difference between variable_scope and name_scope? and What is the difference between variable_ops_scope and variable_scope?.

我仍然不能完全理解他们的区别.我曾尝试将 tf.variable_scope 和 tf.name_scope 用于相同的代码，我发现它们在 TensorBoard 中具有相同的图形.

I still cannot fully understand their differences. I have tried to use tf.variable_scope and tf.name_scope for the same code, I found they have the same graph by TensorBoard.

其他人已经讨论了他们与图表中生成的名称的主要区别，而他们的名称有那么重要吗?我还看到同名的变量会被重用.什么是重用场合?

Other people have discussed their main differences with the generated name in the Graph, while is their name so important? I also saw that the variable with the same name would be reused. What is the reuse occasion?

推荐答案

关键是要理解图中变量和其他张量的区别.任何新创建的张量都将从名称范围中获得前缀.tf.get_variable 将查找没有名称范围修饰符的现有变量.使用 tf.get_variable 新创建的变量仍将增加其名称.

The key is to understand the difference between variables and other tensors in the graph. Any newly created tensors will gain a prefix from a name scope. tf.get_variable will look for existing variables without the name scope modifier. Newly created variables with tf.get_variable will still get their name's augmented.

下面的脚本强调了这些差异.目的是通过将 tf.matmul(x, A) + b 行和变量创建重构为单独的函数 add_layer 来重现 simple 函数>.

The script below highlights these differences. The intention is to reproduce the simple function by refactoring the tf.matmul(x, A) + b line and variable creation into a separate function add_layer.

import tensorflow as tf


def get_x():
    return tf.constant([[1., 2., 3.]], dtype=tf.float32)


def report(out1, out2):
    print(out1.name)
    print(out2.name)
    variables = tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES)
    print([v.name for v in variables])


def simple():
    A = tf.get_variable(shape=(3, 3), dtype=tf.float32, name='A')
    b = tf.get_variable(shape=(3,), dtype=tf.float32, name='b')
    x = get_x()
    out1 = tf.matmul(x, A) + b
    out2 = tf.matmul(out1, A) + b
    return out1, out2


def add_layer(x):
    A = tf.get_variable(shape=(3, 3), dtype=tf.float32, name='A')
    b = tf.get_variable(shape=(3,), dtype=tf.float32, name='b')
    return tf.matmul(x, A) + b


def no_scoping():
    x = get_x()
    out1 = add_layer(x)
    out2 = add_layer(out1)
    return out1, out2


def different_name_scopes():
    x = get_x()
    with tf.name_scope('first_layer'):
        out1 = add_layer(x)
    with tf.name_scope('second_layer'):
        out2 = add_layer(out1)
    return out1, out2


def same_name_scope():
    x = get_x()
    with tf.name_scope('first_layer'):
        out1 = add_layer(x)
    with tf.name_scope('first_layer'):
        out2 = add_layer(out1)
    return out1, out2


def different_variable_scopes():
    x = get_x()
    with tf.variable_scope('first_layer'):
        out1 = add_layer(x)
    with tf.variable_scope('second_layer'):
        out2 = add_layer(out1)
    return out1, out2


def same_variable_scope():
    x = get_x()
    with tf.variable_scope('first_layer'):
        out1 = add_layer(x)
    with tf.variable_scope('first_layer'):
        out2 = add_layer(out1)
    return out1, out2


def same_variable_scope_reuse():
    x = get_x()
    with tf.variable_scope('first_layer'):
        out1 = add_layer(x)
    with tf.variable_scope('first_layer', reuse=True):
        out2 = add_layer(out1)
    return out1, out2


def test_fn(fn, name):
    graph = tf.Graph()
    with graph.as_default():
        try:
            print('****************')
            print(name)
            print('****************')
            out1, out2 = fn()
            report(out1, out2)
            print('----------------')
            print('SUCCESS')
            print('----------------')
        except Exception:
            print('----------------')
            print('FAILED')
            print('----------------')


for fn, name in [
        [simple, 'simple'],
        [no_scoping, 'no_scoping'],
        [different_name_scopes, 'different_name_scopes'],
        [same_name_scope, 'same_name_scope'],
        [different_variable_scopes, 'different_variable_scopes'],
        [same_variable_scope, 'same_variable_scope'],
        [same_variable_scope_reuse, 'same_variable_scope_reuse']
        ]:
    test_fn(fn, name)

结果:

****************
simple
****************
add:0
add_1:0
[u'A:0', u'b:0']
----------------
SUCCESS
----------------
****************
no_scoping
****************
----------------
FAILED
----------------
****************
different_name_scopes
****************
----------------
FAILED
----------------
****************
same_name_scope
****************
----------------
FAILED
----------------
****************
different_variable_scopes
****************
first_layer/add:0
second_layer/add:0
[u'first_layer/A:0', u'first_layer/b:0', u'second_layer/A:0', u'second_layer/b:0']
----------------
SUCCESS
----------------
****************
same_variable_scope
****************
----------------
FAILED
----------------
****************
same_variable_scope_reuse
****************
first_layer/add:0
first_layer_1/add:0
[u'first_layer/A:0', u'first_layer/b:0']
----------------
SUCCESS
----------------

请注意，使用不同的 variable_scope 而不重用不会引发错误，但会创建 A 和 b 的多个副本，这可能会导致不是故意的.

Note that using different variable_scopes without reuse doesn't raise an error, but creates multiple copies of A and b, which may not be intended.

这篇关于variable_scope 和 name_scope 之间的实际区别是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！