如何加载张量流模型并继续训练 [英] how to load a tensorflow model and continue training

问题描述

我想加载一个预训练模型并继续使用该模型进行训练.
保存模型的标准代码片段 (pretrain.py):

I want to load a pretrained model and continue training with this model.
Standard code snippet to save a model (pretrain.py):

tf.reset_default_graph()

# tf Graph input
X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_classes])

mlp_layer_name = ['h1', 'b1', 'h2', 'b2', 'h3', 'b3', 'w_o', 'b_o']
logits = multilayer_perceptron(X, n_input, n_classes, mlp_layer_name)

loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y), name='loss_op')
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
train_op = optimizer.minimize(loss_op, name='train_op')

saver = tf.train.Saver()

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # Training cycle
    for epoch in range(training_epochs):
        avg_cost = 0.

        # Loop over all batches
        for i in range(total_batch):
            batch_x, batch_y = next(train_generator)

            # Run optimization op (backprop) and cost op (to get loss value)
            _, c = sess.run([train_op, loss_op], feed_dict={X: batch_x,
                                                            Y: batch_y})
            # Compute average loss
            avg_cost += c / total_batch

        print("Epoch: {:3d}, cost = {:.6f}".format(epoch+1, avg_cost))

    print("Optimization Finished!")
    saver.save(sess, 'model')
    print("Model saved")

现在加载预训练模型并继续使用它进行训练 (continue.py).

Now load the pretrained model and continue training with it (continue.py).

# tf Graph input
X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_classes])
mlp_layer_name = ['h1', 'b1', 'h2', 'b2', 'h3', 'b3', 'w_o', 'b_o']
logits = multilayer_perceptron(X, n_input, n_classes, mlp_layer_name)
loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y), name='loss_op')
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
train_op = optimizer.minimize(loss_op, name='train_op')

with tf.Session() as sess:
    saver = tf.train.import_meta_graph('model.meta')
    saver.restore(sess, tf.train.latest_checkpoint('./')) # search for checkpoint file

    graph = tf.get_default_graph()

    for epoch in range(training_epochs):
        avg_cost = 0.

        # Loop over all batches
        for i in range(total_batch):
            batch_x, batch_y = next(train_generator)

            # Run optimization op (backprop) and cost op (to get loss value)
            _, c = sess.run([train_op, loss_op], feed_dict={X: batch_x,
                                                            Y: batch_y})
            # Compute average loss
            avg_cost += c / total_batch

        print("Epoch: {:3d}, cost = {:.6f}".format(epoch+1, avg_cost))

但它显示以下错误:

tensorflow.python.framework.errors_impl.FailedPreconditionError:尝试使用未初始化的值 h1[[节点: h1/read = IdentityT=DT_FLOAT, _class=["loc:@h1"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value h1 [[Node: h1/read = IdentityT=DT_FLOAT, _class=["loc:@h1"], _device="/job:localhost/replica:0/task:0/cpu:0"]]

这是我的问题:
1. 在很多 tensorflow 的教程中，它使用 get_tensor_by_name() 来加载权重和偏差.在这里，我不想得到权重和偏差.我只想加载模型并继续使用它进行训练.
2.错误显示张量未初始化.但是，我认为 saver.restore(sess, tf.train.latest_checkpoint('./')) 应该已成功加载权重和偏差.
如果它有助于说明我的问题，这里是 multilayer_perceptron().

Here are my questions:
1. In many tensorflow's tutorial, it uses get_tensor_by_name() to load weights and biases. Here, I don't want to get weights and biases. I just want to load the model and continue training with it.
2. The error showed that tensor is uninitialized. However, I think saver.restore(sess, tf.train.latest_checkpoint('./')) should have loaded the weights and biases succesfully.
Here is multilayer_perceptron() if it helps to illustrate my questoins.

def multilayer_perceptron(x, n_input, n_classes, name):
    n_hidden_1 = 512
    n_hidden_2 = 256
    n_hidden_3 = 128
    # Store layers weight & bias
    weights = {
        'h1' : tf.get_variable(name[0], initializer=tf.random_normal([n_input, n_hidden_1])),
        'h2' : tf.get_variable(name[2], initializer=tf.random_normal([n_hidden_1, n_hidden_2])),
        'h3' : tf.get_variable(name[4], initializer=tf.random_normal([n_hidden_2, n_hidden_3])),
        'w_o': tf.get_variable(name[6], initializer=tf.random_normal([n_hidden_3, n_classes]))
    }
    biases = {
        'b1' : tf.get_variable(name[1], initializer=tf.random_normal([n_hidden_1])),
        'b2' : tf.get_variable(name[3], initializer=tf.random_normal([n_hidden_2])),
        'b3' : tf.get_variable(name[5], initializer=tf.random_normal([n_hidden_3])),
        'b_o': tf.get_variable(name[7], initializer=tf.random_normal([n_classes]))
    }

    layer_1 = tf.nn.relu(tf.add(tf.matmul(x      , weights['h1']), biases['b1']))
    layer_1 = tf.layers.dropout(layer_1, rate=0.5, training=True)
    layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1, weights['h2']), biases['b2']))
    layer_2 = tf.layers.dropout(layer_2, rate=0.3, training=True)
    layer_3 = tf.nn.relu(tf.add(tf.matmul(layer_2, weights['h3']), biases['b3']))
    layer_3 = tf.layers.dropout(layer_3, rate=0.1, training=True)
    out_layer = tf.matmul(layer_3, weights['w_o']) + biases['b_o']
    return out_layer

推荐答案

我想我找到了答案.关键是它不需要调用 tf.train.import_meta_graph() 如果它已经使用了 saver.restore(sess, tf.train.latest_checkpoint('./')).这是我的代码.

I think I found the answer. The key is that it doesn't need to call tf.train.import_meta_graph() if it has already uses saver.restore(sess, tf.train.latest_checkpoint('./')). Here is my code.

# tf Graph input
X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_classes])
mlp_layer_name = ['h1', 'b1', 'h2', 'b2', 'h3', 'b3', 'w_o', 'b_o']
logits = multilayer_perceptron(X, n_input, n_classes, mlp_layer_name)
loss_op = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=Y), name='loss_op')
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
train_op = optimizer.minimize(loss_op, name='train_op')

with tf.Session() as sess:
    saver = tf.train.Saver()
    saver.restore(sess, tf.train.latest_checkpoint('./')) # search for checkpoint file

    graph = tf.get_default_graph()

    for epoch in range(training_epochs):
        avg_cost = 0.

        # Loop over all batches
        for i in range(total_batch):
            batch_x, batch_y = next(train_generator)

            # Run optimization op (backprop) and cost op (to get loss value)
            _, c = sess.run([train_op, loss_op], feed_dict={X: batch_x,
                                                            Y: batch_y})
            # Compute average loss
            avg_cost += c / total_batch

        print("Epoch: {:3d}, cost = {:.6f}".format(epoch+1, avg_cost))

这篇关于如何加载张量流模型并继续训练的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！