尽管重用设置为 true，Tensorflow 仍会创建新变量 [英] Tensorflow creating new variables despite reuse set to true

问题描述

我正在尝试构建一个基本的 RNN，但在训练后尝试使用该网络时出现错误.我在函数 inference

I am trying to build a basic RNN, but I get errors trying to use the network after training. I hold network architecture in a function inference

def inference(inp):
    with tf.name_scope("inference"):
        layer = SimpleRNN(1, activation='sigmoid',   return_sequences=False)(inp)
        layer = Dense(1)(layer)

    return layer 

但每次我调用它时，尽管在训练中使用相同的范围，但会创建另一组变量:

but everytime i call it, another set of variables gets created despite using the same scope in training:

def train(sess, seq_len=2, epochs=100):
    x_input, y_input = generate_data(seq_len)

    with tf.name_scope('train_input'):
        x = tf.placeholder(tf.float32, (None, seq_len, 1))
        y = tf.placeholder(tf.float32, (None, 1))

    with tf.variable_scope('RNN'):
        output = inference(x)

    with tf.name_scope('training'):
        loss = tf.losses.mean_squared_error(labels=y, predictions=output)
        train_op = tf.train.RMSPropOptimizer(learning_rate=0.1).minimize(loss=loss, global_step=tf.train.get_global_step())

    with sess.as_default():
        sess.run([tf.global_variables_initializer(), tf.local_variables_initializer()])

        for i in tqdm.trange(epochs):
            ls, res, _ = sess.run([loss, output, train_op], feed_dict={x:x_input, y:y_input})
            if i%100==0:
                print(f'{ls}: {res[10]} - {y_input[10]}')
            x_input, y_input = generate_data(seq_len)

和预测:

def predict_signal(sess, x, seq_len):   
    # Preparing signal (omitted)
    # Predict
    inp = tf.convert_to_tensor(prepared_signal, tf.float32)
    with sess.as_default():
        with tf.variable_scope('RNN', reuse=True) as scope:
            output = inference(inp)
            result = output.eval()

    return result

我现在已经花了几个小时阅读变量范围，但是在运行预测时我仍然收到错误 Attempting to use uninitialized value RNN_1/inference/simple_rnn_2/kernel，编号为 RNN_1每次调用增加

I have spent couple of hours reading about variables scopes by now, but on running prediction I still get an error Attempting to use uninitialized value RNN_1/inference/simple_rnn_2/kernel, with the number by RNN_1 increasing with each call

推荐答案

这只是推测，直到您向我们展示 SimpleRNN 实现.但是，我怀疑 SimpleRNN 的实现非常糟糕.有一个不同的 getween tf.get_variable 和 tf.Variable.我希望您的 SimpleRNN 使用 tf.Variable.

This is just speculation until you show us the SimpleRNN implementation. However, I suspect that SimpleRNN is very badly implemented. There is a different getween tf.get_variable and tf.Variable. I expect your SimpleRNN to use tf.Variable.

要重现此行为:

import tensorflow as tf


def inference(x):
    w = tf.Variable(1., name='w')
    layer = x + w
    return layer


x = tf.placeholder(tf.float32)

with tf.variable_scope('RNN'):
    output = inference(x)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(output, {x: 10}))

    with sess.as_default():
        with tf.variable_scope('RNN', reuse=True):
            output2 = inference(x)

    print(sess.run(output2, {x: 10}))

这给出了完全相同的错误:

This gives exactly the same error:

尝试使用未初始化的值 RNN_1/w

Attempting to use uninitialized value RNN_1/w

但是使用 w = tf.get_variable('w', initializer=1.) 而不是 w = tf.Variable(1., name='w') 让它工作.

However the version with w = tf.get_variable('w', initializer=1.) instead of w = tf.Variable(1., name='w') makes it work.

为什么?查看文档:

tf.get_variable:

使用这些参数获取现有变量或创建一个新变量.此函数在名称前面加上当前变量范围并执行重用检查.

Gets an existing variable with these parameters or create a new one. This function prefixes the name with the current variable scope and performs reuse checks.

编辑谢谢你的问题(我在你的问题中添加了 keras 标志).这现在成为我最喜欢向人们展示为什么使用 Keras 是他们做过的最糟糕的决定的原因.

edit Thank you for the question (I added the keras flag to your question). This is now becoming my favorite reason to show people why using Keras is the worst decision they ever made.

SimpleRNN

SimpleRNN creates it variables here:

self.kernel = self.add_weight(shape=(input_shape[-1], self.units),
                                      name='kernel',...)

这个executes/a>

This executes the line

weight = K.variable(initializer(shape),
                    dtype=dtype,
                    name=name,
                    constraint=constraint)

其中 结束>

which ends up here

v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)

这是实现中的一个明显缺陷.在 Keras 以正确的方式使用 TensorFlow 之前(至少尊重 scopes 和 variable-collections)，您应该寻找替代方案.有人能给你的最好建议是切换到更好的东西，比如官方 tf.layers.

And this is an obvious flaw in the implementation. Until Keras uses TensorFlow in the correct way (respecting at least scopes and variable-collections), you should look for alternatives. The best advice somebody can give you is to switch to something better like the official tf.layers.

这篇关于尽管重用设置为 true，Tensorflow 仍会创建新变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！