在张量流中初始化变量，变量作用域和import_graph_def [英] Initializing variables, variable scope and import_graph_def in tensorflow

问题描述

在尝试使用import_graph_def进行图形外科手术时，我有许多有关张量流行为的相关问题. 2种不同的图形手术

I have a number of related questions about tensorflow behavior when attempting to do graph surgery using import_graph_def. 2 different graph surgeries

在上图中，我用粗体红色箭头表示2个不同的图形手术.在左侧，有2个图形g1和g2，该手术包括用图形g1中的一个节点及其下面的所有内容替换图形g2中的一个节点. 此帖子.右边的手术涉及替换属于 same 图的节点，我无法弄清楚如何执行，甚至根本不可能.我得到了这个最小的例子

In the image above, I represent with bold red arrows 2 different graph surgeries. On the left, there are 2 graphs, g1 and g2, and the surgery consists of replacing a node in graph g2 by a node - and everything below it - from graph g1. How to do that is explained in this post. The surgery on the right, which involves replacing nodes that belong to the same graph, I haven't been able to figure out how to perform, or even if it is at all possible. I ended up with this minimal example

with tf.Graph().as_default() as g1:
    with tf.variable_scope('foo', reuse=tf.AUTO_REUSE):
        x = tf.placeholder(dtype=tf.float64, shape=[2], name='x')
        c = tf.get_variable('c', initializer=tf.cast(1.0, tf.float64))
        y = tf.identity(2*x, 'y')

        z = tf.identity(3*x*c, 'z')

        g1_def = g1.as_graph_def()
        z1, = tf.import_graph_def(g1_def, input_map={'foo/x:0' : y}, return_elements=["foo/z:0"],
                              name='z1')
        init_op = tf.global_variables_initializer()
        print(tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES, scope='foo'))


with tf.Session(graph=g1) as sess:
    sess.run(init_op)
    print(sess.run(z, feed_dict={'foo/x:0' : np.array([1.0, 2.0])}) )
    print(sess.run(tf.report_uninitialized_variables()))
    # z1 = sess.run(z1, feed_dict={'foo/x:0' : np.array([1.0, 2.0])})

此代码按原样运行. 3张照片分别产生:

This code runs as it is. The 3 prints yield respectively:

[<tf.Variable 'foo/c:0' shape=() dtype=float64_ref>]
[ 3.  6.]
[]

特别是，最后打印显示没有未初始化的变量.但是，取消注释最后一行会产生错误

In particular, the last print informs that there are no unintialized variables. However, uncommenting the last line, yields the error

FailedPreconditionError (see above for traceback): Attempting to use uninitialized value foo/z1/foo/c

请注意，如果我从上面的z的定义中删除了c，这也将有效.但是，我想了解这个错误.首先，为什么将变量报告为foo/z1/foo/c?为什么示波器foo出现两次?为什么在打印未初始化的变量时什么也没报告?当我在范围foo下打印GLOBAL_VARIABLES集合时，为什么仅报告foo/c?

Note that if I remove c from the definition of z above, this would also work. However, I would like to understand this error. To begin with, why is the variable reported as foo/z1/foo/c? Why does the scope foo appear twice? Why is nothing reported when I print the uninitialized variables? Why is only foo/c reported when I print the GLOBAL_VARIABLES collection under the scope foo?

PS:我想有一种更简单的方法来问这个问题，即

PS: I guess that there is a simpler way to ask the question which is, what is the tensorflow analogue of

theano.clone(some_tensor, replace={input_var : replace_var})

推荐答案

首先，为什么将变量报告为foo/z1/foo/c? 为什么作用域foo出现两次?

To begin with, why is the variable reported as foo/z1/foo/c? Why does the scope foo appear twice?

调用tf.import_graph_def(...)后，图形将重复.第一张图以foo分数定义.第二个子图已在范围foo/z1下导入(因为在上面的范围内保留了name='z1'和foo).因此，图g1现在包含以下张量:

After you've called tf.import_graph_def(...), the graph got duplicated. The first graph is defined in foo score. The second subgraph has been imported under the scope foo/z1 (because name='z1', plus foo is preserved from the scope above). So the graph g1 now contains the following tensors:

foo/x
foo/y
foo/c
...
foo/z1/foo/x
foo/z1/foo/y
foo/z1/foo/c
...

第一个foo/c已初始化，但第二个foo/z1/foo/c未初始化(请参见下文).

The first foo/c is initialized, but the second foo/z1/foo/c is not (see below).

为什么在打印未初始化的变量时没有任何报告?在范围foo下打印GLOBAL_VARIABLES集合时，为什么仅报告foo/c?

Why is nothing reported when I print the uninitialized variables? Why is only foo/c reported when I print the GLOBAL_VARIABLES collection under the scope foo?

由于默认情况下report_uninitialized_variables()扫描LOCAL_VARIABLES和GLOBAL_VARIABLES，因此这基本上是相同的问题.

Since report_uninitialized_variables() scans LOCAL_VARIABLES and GLOBAL_VARIABLES by default, this is basically the same question.

而且可能是一个错误:tf.import_graph_def调用后GLOBAL_VARIABLES集合未更新.我说这可能是因为GLOBAL_VARIABLES只是为了方便收集而设计的. Tensorflow试图使其保持最新状态，但可能不能保证它始终具有所有变量. tf.add_to_collection 的事实公开支持这一想法-可以添加他们想要的任何收藏都可以得到任何价值.最重要的是:此行为在将来的版本中可能会或可能不会更改，但是从1.5版本开始，客户端负责在图形导入后更新全局变量.

And it probably is a bug: GLOBAL_VARIABLES collection isn't updated after tf.import_graph_def call. I say probably because GLOBAL_VARIABLES was designed as a mere convenience collection. Tensorflow tries to keep it up do date, but probably doesn't guarantee it always has all variables. The fact that tf.add_to_collection exists publicly supports this idea -- one can add any value to any collection if they want it. Bottom line: this behavior may or may not change in future versions, but as of 1.5 the client is responsible to update the global variables after graph import.

特别是，最后打印显示没有未初始化的变量.但是，取消注释最后一行会产生错误

In particular, the last print informs that there are no unintialized variables. However, uncommenting the last line, yields the error

要解决此错误，您只需要为z1子图运行初始化程序.像这样:

To fix this error, you simply need to run the initializer for the z1 subgraph. Like this:

# note that it's defined before `g1.as_graph_def()` to be a part of graph def
init_op = tf.global_variables_initializer()

g1_def = g1.as_graph_def()
z1, = tf.import_graph_def(g1_def, input_map={'foo/x:0': y}, return_elements=["foo/z:0"],
                          name='z1')

# find the init op
z1_init_op = tf.get_default_graph().get_operation_by_name('foo/z1/foo/init')

...

sess.run(z1_init_op)

瞧！就像您想要的那样，您具有重复的图.

And voila! You have the duplicated graphs, just like you wanted to.

这篇关于在张量流中初始化变量，变量作用域和import_graph_def的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！