恢复模型需要很长时间 [英] Restoring model take to long

问题描述

我在重新串接我的模型时遇到问题.我训练模型并使用此代码保存模型.我不确定这是否是正确的方法，我将不胜感激.当我尝试恢复模型时出现问题.我只需要它来预测，它不会被进一步训练.从模型中恢复参数需要很长时间.我如何改进我的模型保护程序或模型恢复程序以使其快速，假设我只需要它用于预测.

I have a problem restring my model. I trained model and saved a model using this code. I'm not really sure if this is the proper method I would be grateful for suggestions. The problem occurs when I'm trying to restore model. I need it only to predict, it won't be furder trained. It takes forever to restore parameters from the model. How can I improve my model saver or model restorer to make it quick, under the assumption I need it only for predicting.

X = tf.placeholder(tf.float32, [None, 56, 56, 1])
Y_ = tf.placeholder(tf.float32, [None, 36])

L1 = 432
L2 = 72
L3 = 36

W1 = tf.Variable(tf.truncated_normal([3136, L1], stddev=0.1))
b1 = tf.Variable(tf.zeros([L1]))
W2 = tf.Variable(tf.truncated_normal([L1, L2], stddev=0.1))
b2 = tf.Variable(tf.zeros([L2]))
W3 = tf.Variable(tf.truncated_normal([L2, L3], stddev=0.1))
b3 = tf.Variable(tf.zeros([L3]))

XX = tf.reshape(X, [-1, 3136])

Y1 = tf.nn.sigmoid(tf.matmul(XX, W1) + b1)
Y1 = tf.nn.dropout(Y1, keep_prob=0.8)
Y2 = tf.nn.sigmoid(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, keep_prob=0.8)
Ylogits = tf.matmul(Y2, W3) + b3
Y = tf.nn.softmax(Ylogits)

cross_entropy = tf.nn.softmax_cross_entropy_with_logits_v2(logits=Ylogits, labels=Y_)
cross_entropy = tf.reduce_mean(cross_entropy) * 100
correct_prediction = tf.equal(tf.argmax(Y, 1), tf.argmax(Y_, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
train_step = tf.train.GradientDescentOptimizer(0.0001).minimize(cross_entropy)

allweights = tf.concat([tf.reshape(W1, [-1]), tf.reshape(W2, [-1]), tf.reshape(W3, [-1])], 0)
allbiases = tf.concat([tf.reshape(b1, [-1]), tf.reshape(b2, [-1]), tf.reshape(b3, [-1])], 0)

init = tf.global_variables_initializer()

saver = tf.train.Saver()

def next_batch(x, y, batch, step):
    x_temp = x[cur_step:(step+batch)]
    y_temp = np.squeeze(y[step:(step + batch)])
    return x_temp, y_temp


with tf.Session() as sess:
    sess.run(init)
    cur_step = 0
    for i in range(NUM_ITERS + 1):
        batch_X, batch_Y = next_batch(train_xx, train_yy, BATCH, cur_step)
        if i % DISPLAY_STEP == 0:
            acc_trn, loss_trn, w, b = sess.run([accuracy, cross_entropy, allweights, allbiases], feed_dict={X: batch_X, Y_: batch_Y})
            acc_tst, loss_tst = sess.run([accuracy, cross_entropy], feed_dict={X: test_xx, Y_: test_yy})

        sess.run(train_step, feed_dict={X: batch_X, Y_: batch_Y})
    save_path = saver.save(sess, "abc/model")

恢复:

X = tf.placeholder(tf.float32, [None, 56, 56, 1])
Y_ = tf.placeholder(tf.float32, [None, 36])

L1 = 432
L2 = 72
L3 = 36

W1 = tf.Variable(tf.truncated_normal([3136, L1], stddev=0.1))
b1 = tf.Variable(tf.zeros([L1]))

W2 = tf.Variable(tf.truncated_normal([L1, L2], stddev=0.1))
b2 = tf.Variable(tf.zeros([L2]))

W3 = tf.Variable(tf.truncated_normal([L2, L3], stddev=0.1))
b3 = tf.Variable(tf.zeros([L3]))


XX = tf.reshape(X, [-1, 3136])

Y1 = tf.nn.sigmoid(tf.matmul(XX, W1) + b1)
Y1 = tf.nn.dropout(Y1, keep_prob=0.8)
Y2 = tf.nn.sigmoid(tf.matmul(Y1, W2) + b2)
Y2 = tf.nn.dropout(Y2, keep_prob=0.8)
Ylogits = tf.matmul(Y2, W3) + b3
Y = tf.nn.softmax(Ylogits)

with tf.Session() as sess:
    saver = tf.train.Saver()
    saver = tf.train.import_meta_graph('model.meta')
    saver.restore(sess, 'model')

也许模型是使用 Google Colab 的 GPU 进行训练并且我正在我的 PC 上恢复它的事实很重要.

Maybe a fact that model is trained using Google Colab's GPU and I'm restoring it on my PC is important.

推荐答案

它的副本:Tensorflow:如何保存/恢复模型?.

您保存模型是正确的，但不是您的恢复.您正在做的是尝试创建一个与保存的模型具有相同节点的新图形，而不是从保存的图形中恢复它.以下步骤应该可以解决您关于如何恢复模型的问题:

Your saving of the model is right but not your restoring. What you are doing is trying to create a new graph with the same nodes as the saved model, instead of restoring it from the saved graph. The following steps should address your issue on how to restore a model:

#Start with resetting the default graph
tf.reset_default_graph()

with tf.Session() as sess:

   # Nodes:Before loading the graph
   print([n.name for n in tf.get_default_graph().as_graph_def().node])
   # Output is [] as no graph is loaded yet.

   # First let's load meta graph 
   saver = tf.train.import_meta_graph("abc/model.meta")
   # Nodes:after loading the graph'
   print([n.name for n in tf.get_default_graph().as_graph_def().node])
   # Output is [save/RestoreV2/shape_and_slices', 'save/RestoreV2/tensor_ ...]

   # The above step doesnt load the weights, can be checked by
   print(sess.run('Variable_1:0'))
   # Error: attempting to use uninitialized graph.

   #load the weights 
   saver.restore(sess,tf.train.latest_checkpoint('./abc/'))
   print(sess.run('Variable_1:0'))
   # Output: [-2.80421402e-04  3.53254407e-04 ...]

现在我们已经加载并准备好了节点，您需要访问其中的一些进行推理.但是由于节点没有正确命名，因此很难确定哪个节点是输入和输出.为了避免这种情况，您需要在使用 name 参数正确保存模型时 name 张量/操作，例如:

Now we have the nodes loaded and ready, you need to access some of them for inference. But since the nodes are not named properly, its not easy to figure out which node are inputs and outputs. To avoid this you need to name the tensors/ops when saving the model properly using the name argument like:

X = tf.placeholder(tf.float32, [None, 56, 56, 1], name='X')
Y = tf.identity(f.nn.softmax(Ylogits), name='logits').

在加载了图形和权重后的推理图中，您可以使用 get_tensor_by_name 获取这些张量:

In your inference graph once you have loaded the graph and weights, you can get these tensors using get_tensor_by_name:

with tf.Session() as sess:

   #Load the graph and weights as above
   ....

   graph = tf.get_default_graph()
   X_infer = graph.get_tensor_by_name('X:0')
   Y_infer = graph.get_tensor_by_name('logits:0')
   sess.run(Y_infer,{X_infer:new_input}

这篇关于恢复模型需要很长时间的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！