Tensorflow 恢复权重未设置 [英] Tensorflow restored weights is not set

问题描述

我正在尝试在 Tensorflow 中恢复我训练过的模型.问题是权重似乎没有正确恢复.

I'm trying to restore a model in Tensorflow which I've trained. The problem is that it does not seem like the weights are properly restored.

对于训练，我将权重和偏差定义为:

For the training I've got the weights and biases defined as:

W = {
   'h1': tf.Variable(tf.random_normal([n_inputs, n_hidden_1]), name='wh1'),
   'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2]), name='wh2'),
   'o': tf.Variable(tf.random_normal([n_hidden_2, n_classes]), name='wo')
}
b = {
   'b1': tf.Variable(tf.random_normal([n_hidden_1]), name='bh1'),
   'b2': tf.Variable(tf.random_normal([n_hidden_2]), name='bh2'),
   'o': tf.Variable(tf.random_normal([n_classes]), name='bo')
}

然后我对自己的自定义 2D 图像数据集进行一些训练，并通过调用 tf.saver

Then I do some training on my own custom 2D image dataset and save the model by calling the tf.saver

saver = tf.train.Saver()
saver.save(sess, 'tf.model')

稍后我想用完全相同的权重恢复该模型，所以我像以前一样构建模型(也使用 random_normal 初始化)并调用 tf.saver.restore

Later I want to restore that model with the exact same weights, so I build the model as before (also with the random_normal initialization) and call the tf.saver.restore

saver = tf.train.import_meta_graph('tf.model.meta')
saver.restore(sess, tf.train.latest_checkpoint('./'))

现在，如果我打电话:

temp = sess.run(W['h1'][0][0])
print temp

我得到的是随机值，而不是权重的恢复值.

I get random values, and not the restored value of the weight.

我在这个上画了一个空白，有人能指出我正确的方向吗?

I've drawn a blank on this one, can somebody point me in the right direction?

仅供参考，我尝试过(没有)运气来简单地声明 tf.Variable s，但我一直得到:

FYI, I've tried (without) luck to simply declare the tf.Variables, but I keep getting:

ValueError: initial_value must be specified.

尽管 Tensorflow 自己声明应该可以简单地声明没有初始值(https://www.tensorflow.org/programmers_guide/variables 部分:恢复值)

even though Tensorflow themselves state that it should be possible to simply declare with no initial value (https://www.tensorflow.org/programmers_guide/variables part: Restoring Values)

更新 1

当我按照建议运行时

all_vars = tf.global_variables()
for v in all_vars:
   print v.name

我得到以下输出:

wh1:0
wh2:0
wo:0
bh1:0
bh2:0
bo:0
wh1:0
wh2:0
wo:0
bh1:0
bh2:0
bo:0
beta1_power:0
beta2_power:0
wh1/Adam:0
wh1/Adam_1:0
wh2/Adam:0
wh2/Adam_1:0
wo/Adam:0
wo/Adam_1:0
bh1/Adam:0
bh1/Adam_1:0
bh2/Adam:0
bh2/Adam_1:0
bo/Adam:0
bo/Adam_1:0

这表明确实读取了变量.但是调用

Which shows that the variables indeed is read. However invoking

print sess.run("wh1:0")

导致错误:Attempting to use uninitialized value wh1

Results in the error: Attempting to use uninitialized value wh1

推荐答案

所以在你们的帮助下，我最终将程序的保存和恢复部分分成了两个文件，以确保没有初始化不需要的变量.

So with the help of you guys, I ended up dividing the saving and restoring parts of my program into two files, to ensure that no unwanted variables were initialized.

训练和保存例程fnn.py

def build(self, topology):
    """
    Builds the topology of the model
    """

    # Sanity check
    assert len(topology) == 4

    n_inputs = topology[0]
    n_hidden_1 = topology[1]
    n_hidden_2 = topology[2]
    n_classes = topology[3]

    # Sanity check
    assert self.img_h * self.img_w == n_inputs

    # Instantiate TF Placeholders
    self.x = tf.placeholder(tf.float32, [None, n_inputs], name='x')
    self.y = tf.placeholder(tf.float32, [None, n_classes], name='y')
    self.W = {
        'h1': tf.Variable(tf.random_normal([n_inputs, n_hidden_1]), name='wh1'),
        'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2]), name='wh2'),
        'o': tf.Variable(tf.random_normal([n_hidden_2, n_classes]), name='wo')
    }
    self.b = {
        'b1': tf.Variable(tf.random_normal([n_hidden_1]), name='bh1'),
        'b2': tf.Variable(tf.random_normal([n_hidden_2]), name='bh2'),
        'o': tf.Variable(tf.random_normal([n_classes]), name='bo')
    }

    # Create model
    self.l1 = tf.nn.sigmoid(tf.add(tf.matmul(self.x, self.W['h1']), self.b['b1']))
    self.l2 = tf.nn.sigmoid(tf.add(tf.matmul(self.l1, self.W['h2']), self.b['b2']))
    logits = tf.add(tf.matmul(self.l2, self.W['o']), self.b['o'])

    # Define predict operation
    self.predict_op = tf.argmax(logits, 1)
    probs = tf.nn.softmax(logits, name='probs')

    # Define cost function
    self.cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits, self.y))

    # Adding these to collection so we can restore them again
    tf.add_to_collection('inputs', self.x)
    tf.add_to_collection('inputs', self.y)
    tf.add_to_collection('outputs', logits)
    tf.add_to_collection('outputs', probs)
    tf.add_to_collection('outputs', self.predict_op)

def train(self, X, Y, n_epochs=10, learning_rate=0.001, logs_path=None):
    """
    Trains the Model
    """
    self.optimizer = tf.train.AdamOptimizer(learning_rate).minimize(self.cost)

    costs = []

    # Instantiate TF Saver
    saver = tf.train.Saver()

    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        sess.run(tf.local_variables_initializer())

        # start the threads used for reading files
        coord = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(sess=sess, coord=coord)

        # Compute total number of batches
        total_batch = int(self.get_num_examples() / self.batch_size)

        # start training
        for epoch in range(n_epochs):
            for i in range(total_batch):

                batch_xs, batch_ys = sess.run([X, Y])

                # run the training step with feed of images
                _, cost = sess.run([self.optimizer, self.cost], feed_dict={self.x: batch_xs,
                                                                           self.y: batch_ys})
                costs.append(cost)
                print "step %d" % (epoch * total_batch + i)
            #costs.append(cost)
            print "Epoch %d" % epoch

        saver.save(sess, self.model_file)

        temp = sess.run(self.W['h1'][0][0])
        print temp

        if self.visu:
            plt.plot(costs)
            plt.show()

        # finalize
        coord.request_stop()
        coord.join(threads)

预测例程fnn_eval.py:

with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        sess.run(tf.local_variables_initializer())

        g = tf.get_default_graph()

        # restore the model
        self.saver = tf.train.import_meta_graph(self.model_file)
        self.saver.restore(sess, tf.train.latest_checkpoint('./tfmodels/fnn/'))

        wh1 = g.get_tensor_by_name("wh1:0")
        print sess.run(wh1[0][0])

        x, y = tf.get_collection('inputs')
        logits, probs, predict_op = tf.get_collection('outputs')

        coord = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(sess=sess, coord=coord)

        predictions = []

        print Y.eval()

        for i in range(1):#range(self.get_num_examples()):
            batch_xs = sess.run(X)
            # Reshape batch_xs if only a single image is given
            #   (numpy is 4D: batch_size * heigth * width * channels)
            batch_xs = np.reshape(batch_xs, (-1, self.img_w * self.img_h))
            prediction, probabilities, logit = sess.run([predict_op, probs, logits], feed_dict={x: batch_xs})
            predictions.append(prediction[0])

        # finalize
        coord.request_stop()
        coord.join(threads)

这篇关于Tensorflow 恢复权重未设置的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！