了解 Tensorflow 中的可变范围示例 [英] Understanding Variable scope example in Tensorflow

问题描述

我正在查看 Tensorflow 的机制部分，特别是在 共享变量.在问题"部分，他们正在处理卷积神经网络，并提供以下代码(通过模型运行图像):

I was looking at the mechanics section for Tensorflow, specifically on shared variables. In the section "The problem", they are dealing with a convolutional neural net, and provide the following code (which runs an image through the model):

# First call creates one set of variables.
result1 = my_image_filter(image1)
# Another set is created in the second call.
result2 = my_image_filter(image2)

如果模型以这种方式实现，那么是否无法学习/更新参数，因为我的训练集中的每个图像都有一组新的参数?

If the model was implemented in such a way, would it then be impossible to learn/update the parameters because there's a new set of parameters for each image in my training set?

我还尝试了简单线性回归的问题"方法 example，并且这种实现方法似乎没有任何问题.正如代码的最后一行所显示的那样，培训似乎也有效.所以我想知道 tensorflow 文档和我在做什么之间是否存在细微的差异.:

I've also tried "the problem" approach on a simple linear regression example, and there do not appear to be any issues with this method of implementation. Training seems to work as well as can be shown by the last line of the code. So I'm wondering if there is a subtle discrepancy in the tensorflow documentation and what I'm doing. :

import tensorflow as tf
import numpy as np

trX = np.linspace(-1, 1, 101)
trY = 2 * trX + np.random.randn(*trX.shape) * 0.33 # create a y value which is         approximately linear but with some random noise

X = tf.placeholder("float") # create symbolic variables
Y = tf.placeholder("float")


def model(X):
    with tf.variable_scope("param"):
        w = tf.Variable(0.0, name="weights") # create a shared variable (like theano.shared) for the weight matrix

    return tf.mul(X, w) # lr is just X*w so this model line is pretty simple


y_model = model(X)

cost = (tf.pow(Y-y_model, 2)) # use sqr error for cost function

train_op = tf.train.GradientDescentOptimizer(0.01).minimize(cost) # construct an optimizer to minimize cost and fit line to my data

sess = tf.Session()
init = tf.initialize_all_variables() # you need to initialize variables (in this case just variable W)
sess.run(init)

with tf.variable_scope("train"):
    for i in range(100):
        for (x, y) in zip(trX, trY):
        sess.run(train_op, feed_dict={X: x, Y: y})

print sess.run(y_model, feed_dict={X: np.array([1,2,3])})

推荐答案

每个整个训练(和测试)集只需创建一次变量集.变量作用域的目标是允许对参数子集进行模块化，例如属于层的那些(例如，当一个层的架构重复时，相同的名称可以是在每个层范围内使用).

One has to create the variable set only once per whole training (and testing) set. The goal of variable scopes is to allow for modularization of subsets of parameters, such as those belonging to layers (e.g. when architecture of a layer is repeated, the same names can be used within each layer scope).

在您的示例中，您仅在 model 函数中创建参数.您可以打印出您的变量名称以查看它是否已分配给指定的范围:

In your example you create parameters only in the model function. You can print out your variable names to see that it is assigned to the specified scope:

from __future__ import print_function

X = tf.placeholder("float") # create symbolic variables
Y = tf.placeholder("float")
print("X:", X.name)
print("Y:", Y.name)

def model(X):
    with tf.variable_scope("param"):
        w = tf.Variable(0.0, name="weights") # create a shared variable (like theano.shared) for the weight matrix
    print("w:", w.name)
    return tf.mul(X, w) 

对 sess.run(train_op, feed_dict={X: x, Y: y}) 的调用仅评估 train_op 的值，给定 train_op 的值code>X 和 Y.那里没有创建新变量(包括参数)；因此，它没有效果.您可以通过再次打印出来来确保变量名称保持不变:

The call to sess.run(train_op, feed_dict={X: x, Y: y}) only evaluates the value of train_op given the provided values of X and Y. No new variables (incl. parameters) are created there; therefore, it has no effect. You can make sure the variable names stay the same by again printing them out:

with tf.variable_scope("train"):
    print("X:", X.name)
    print("Y:", Y.name)
    for i in range(100):
        for (x, y) in zip(trX, trY):
            sess.run(train_op, feed_dict={X: x, Y: y})

您会看到变量名保持不变，因为它们已经被初始化.

You will see that variable names stay the same, as they are already initialized.

如果您想检索变量使用其范围，您需要在 tf.variable_scope 附件中使用 get_variable:

If you'd like to retrieve a variable using its scope, you need to use get_variable within a tf.variable_scope enclosure:

with tf.variable_scope("param"):
    w = tf.get_variable("weights", [1])
print("w:", w.name)

这篇关于了解 Tensorflow 中的可变范围示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！