在 Tensorflow 中没有保存错误的变量 [英] No variable to save error in Tensorflow

问题描述

我正在尝试保存模型，然后将其重新用于对我的图像进行分类，但不幸的是，我在恢复已保存的模型时出错.

I am trying to save the model and then reuse it for classifying my images but unfortunately i am getting errors in restoring the model that i have saved.

创建模型的代码:

# Deep Learning
# =============
# 
# Assignment 4
# ------------

# In[25]:

# These are all the modules we'll be using later. Make sure you can import them
# before proceeding further.
from __future__ import print_function
import numpy as np
import tensorflow as tf
from six.moves import cPickle as pickle
from six.moves import range


# In[37]:

pickle_file = 'notMNIST.pickle'

with open(pickle_file, 'rb') as f:
  save = pickle.load(f)
  train_dataset = save['train_dataset']
  train_labels = save['train_labels']
  valid_dataset = save['valid_dataset']
  valid_labels = save['valid_labels']
  test_dataset = save['test_dataset']
  test_labels = save['test_labels']
  del save  # hint to help gc free up memory
  print('Training set', train_dataset.shape, train_labels.shape)
  print('Validation set', valid_dataset.shape, valid_labels.shape)
  print('Test set', test_dataset.shape, test_labels.shape)
  print(test_labels)


# Reformat into a TensorFlow-friendly shape:
# - convolutions need the image data formatted as a cube (width by height by #channels)
# - labels as float 1-hot encodings.

# In[38]:

image_size = 28
num_labels = 10
num_channels = 1 # grayscale

import numpy as np

def reformat(dataset, labels):
  dataset = dataset.reshape(
    (-1, image_size, image_size, num_channels)).astype(np.float32)
  #print(np.arange(num_labels))
  labels = (np.arange(num_labels) == labels[:,None]).astype(np.float32)
  #print(labels[0,:])
  print(labels[0])
  return dataset, labels
train_dataset, train_labels = reformat(train_dataset, train_labels)
valid_dataset, valid_labels = reformat(valid_dataset, valid_labels)
test_dataset, test_labels = reformat(test_dataset, test_labels)
print('Training set', train_dataset.shape, train_labels.shape)
print('Validation set', valid_dataset.shape, valid_labels.shape)
print('Test set', test_dataset.shape, test_labels.shape)
#print(labels[0])


# In[39]:

def accuracy(predictions, labels):
  return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1))
          / predictions.shape[0])


# Let's build a small network with two convolutional layers, followed by one fully connected layer. Convolutional networks are more expensive computationally, so we'll limit its depth and number of fully connected nodes.

# In[47]:

batch_size = 16
patch_size = 5
depth = 16
num_hidden = 64

graph = tf.Graph()

with graph.as_default():

  # Input data.
  tf_train_dataset = tf.placeholder(
    tf.float32, shape=(batch_size, image_size, image_size, num_channels))
  tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
  tf_valid_dataset = tf.constant(valid_dataset)
  tf_test_dataset = tf.constant(test_dataset)

  # Variables.
  layer1_weights = tf.Variable(tf.truncated_normal(
      [patch_size, patch_size, num_channels, depth], stddev=0.1),name="layer1_weights")
  layer1_biases = tf.Variable(tf.zeros([depth]),name = "layer1_biases")
  layer2_weights = tf.Variable(tf.truncated_normal(
      [patch_size, patch_size, depth, depth], stddev=0.1),name = "layer2_weights")
  layer2_biases = tf.Variable(tf.constant(1.0, shape=[depth]),name ="layer2_biases")
  layer3_weights = tf.Variable(tf.truncated_normal(
      [image_size // 4 * image_size // 4 * depth, num_hidden], stddev=0.1),name="layer3_biases")
  layer3_biases = tf.Variable(tf.constant(1.0, shape=[num_hidden]),name = "layer3_biases")
  layer4_weights = tf.Variable(tf.truncated_normal(
      [num_hidden, num_labels], stddev=0.1),name = "layer4_weights")
  layer4_biases = tf.Variable(tf.constant(1.0, shape=[num_labels]),name = "layer4_biases")

  # Model.
  def model(data):
    conv = tf.nn.conv2d(data, layer1_weights, [1, 2, 2, 1], padding='SAME')
    hidden = tf.nn.relu(conv + layer1_biases)
    conv = tf.nn.conv2d(hidden, layer2_weights, [1, 2, 2, 1], padding='SAME')
    hidden = tf.nn.relu(conv + layer2_biases)
    shape = hidden.get_shape().as_list()
    reshape = tf.reshape(hidden, [shape[0], shape[1] * shape[2] * shape[3]])
    hidden = tf.nn.relu(tf.matmul(reshape, layer3_weights) + layer3_biases)
    return tf.matmul(hidden, layer4_weights) + layer4_biases

  # Training computation.
  logits = model(tf_train_dataset)
  loss = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(logits, tf_train_labels))

  # Optimizer.
  optimizer = tf.train.GradientDescentOptimizer(0.05).minimize(loss)

  # Predictions for the training, validation, and test data.
  train_prediction = tf.nn.softmax(logits)
  valid_prediction = tf.nn.softmax(model(tf_valid_dataset))
  test_prediction = tf.nn.softmax(model(tf_test_dataset))


# In[48]:

num_steps = 1001
#saver = tf.train.Saver()
with tf.Session(graph=graph) as session:
  tf.initialize_all_variables().run()
  print('Initialized')
  for step in range(num_steps):
    offset = (step * batch_size) % (train_labels.shape[0] - batch_size)
    batch_data = train_dataset[offset:(offset + batch_size), :, :, :]
    batch_labels = train_labels[offset:(offset + batch_size), :]
    feed_dict = {tf_train_dataset : batch_data, tf_train_labels : batch_labels}
    _, l, predictions = session.run(
      [optimizer, loss, train_prediction], feed_dict=feed_dict)
    if (step % 50 == 0):
      print('Minibatch loss at step %d: %f' % (step, l))
      print('Minibatch accuracy: %.1f%%' % accuracy(predictions, batch_labels))
      print('Validation accuracy: %.1f%%' % accuracy(
        valid_prediction.eval(), valid_labels))
  print('Test accuracy: %.1f%%' % accuracy(test_prediction.eval(), test_labels))
  save_path = tf.train.Saver().save(session, "/tmp/model.ckpt")
  print("Model saved in file: %s" % save_path)

一切正常，模型存储在各自的文件夹中.

Everything works fine and the model is stored in the respective folder .

我又创建了一个 python 文件，我尝试在其中恢复模型，但在那里出现错误

# In[1]:
from __future__ import print_function
import numpy as np
import tensorflow as tf
from six.moves import cPickle as pickle
from six.moves import range


# In[3]:

image_size = 28
num_labels = 10
num_channels = 1 # grayscale
import numpy as np


# In[4]:

def accuracy(predictions, labels):
  return (100.0 * np.sum(np.argmax(predictions, 1) == np.argmax(labels, 1))
          / predictions.shape[0])


# In[8]:

batch_size = 16
patch_size = 5
depth = 16
num_hidden = 64

graph = tf.Graph()

with graph.as_default():

  '''# Input data.
  tf_train_dataset = tf.placeholder(
    tf.float32, shape=(batch_size, image_size, image_size, num_channels))
  tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
  tf_valid_dataset = tf.constant(valid_dataset)
  tf_test_dataset = tf.constant(test_dataset)'''

  # Variables.
  layer1_weights = tf.Variable(tf.truncated_normal(
      [patch_size, patch_size, num_channels, depth], stddev=0.1),name="layer1_weights")
  layer1_biases = tf.Variable(tf.zeros([depth]),name = "layer1_biases")
  layer2_weights = tf.Variable(tf.truncated_normal(
      [patch_size, patch_size, depth, depth], stddev=0.1),name = "layer2_weights")
  layer2_biases = tf.Variable(tf.constant(1.0, shape=[depth]),name ="layer2_biases")
  layer3_weights = tf.Variable(tf.truncated_normal(
      [image_size // 4 * image_size // 4 * depth, num_hidden], stddev=0.1),name="layer3_biases")
  layer3_biases = tf.Variable(tf.constant(1.0, shape=[num_hidden]),name = "layer3_biases")
  layer4_weights = tf.Variable(tf.truncated_normal(
      [num_hidden, num_labels], stddev=0.1),name = "layer4_weights")
  layer4_biases = tf.Variable(tf.constant(1.0, shape=[num_labels]),name = "layer4_biases")

  # Model.
  def model(data):
    conv = tf.nn.conv2d(data, layer1_weights, [1, 2, 2, 1], padding='SAME')
    hidden = tf.nn.relu(conv + layer1_biases)
    conv = tf.nn.conv2d(hidden, layer2_weights, [1, 2, 2, 1], padding='SAME')
    hidden = tf.nn.relu(conv + layer2_biases)
    shape = hidden.get_shape().as_list()
    reshape = tf.reshape(hidden, [shape[0], shape[1] * shape[2] * shape[3]])
    hidden = tf.nn.relu(tf.matmul(reshape, layer3_weights) + layer3_biases)
    return tf.matmul(hidden, layer4_weights) + layer4_biases

  '''# Training computation.
  logits = model(tf_train_dataset)
  loss = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(logits, tf_train_labels))

  # Optimizer.
  optimizer = tf.train.GradientDescentOptimizer(0.05).minimize(loss)'''

  # Predictions for the training, validation, and test data.
  #train_prediction = tf.nn.softmax(logits)
  #valid_prediction = tf.nn.softmax(model(tf_valid_dataset))
  #test_prediction = tf.nn.softmax(model(tf_test_dataset))

# In[17]:

#saver = tf.train.Saver()
with tf.Session() as sess:
  # Restore variables from disk.
  tf.train.Saver().restore(sess, "/tmp/model.ckpt")
  print("Model restored.")
  # Do some work with the model

我得到的错误是:

没有要保存的变量

任何帮助将不胜感激

推荐答案

这里的错误很微妙.在 In[8] 中，您创建了一个 tf.Graph 调用了 graph 并将其设置为 with graph.as_default(): 块.这意味着所有的变量都是在 graph 中创建的，如果你打印 graph.all_variables() 你应该看到你的变量列表.

The error here is quite subtle. In In[8] you create a tf.Graph called graph and set it as default for the with graph.as_default(): block. This means that all of the variables are created in graph, and if you print graph.all_variables() you should see a list of your variables.

但是，您在创建 (i) tf.Session，以及 (ii) tf.train.Saver.这意味着会话和保护程序是在不同的图形中创建的(全局默认tf.Graph，当您没有明确创建一个并将其设置为默认值时使用)，它不包含任何变量——或根本不包含任何节点.

However, you exit the with block before creating (i) the tf.Session, and (ii) the tf.train.Saver. This means that the session and saver are created in a different graph (the global default tf.Graph that is used when you don't explicitly create one and set it as default), which doesn't contain any variables—or any nodes at all.

至少有两种解决方案:

1. 正如 Yaroslav 建议的，您可以在不使用 和 graph.as_default(): 块，避免了多图的混淆.但是，这可能会导致 IPython 笔记本中不同单元格之间的名称冲突，这在使用 tf.train.Saver 时很尴尬，因为它使用了tf.Variable 作为检查点文件中的键.

1. As Yaroslav suggests, you can write your program without using the with graph.as_default(): block, which avoids the confusion with multiple graphs. However, this can lead to name collisions between different cells in your IPython notebook, which is awkward when using the tf.train.Saver, since it uses the name property of a tf.Variable as the key in the checkpoint file.

您可以在 with graph.as_default(): 块中创建保护程序inside，并创建 tf.Session一个显式图，如下:

You can create the saver inside the with graph.as_default(): block, and create the tf.Session with an explicit graph, as follows:

with graph.as_default():
    # [Variable and model creation goes here.]

    saver = tf.train.Saver()  # Gets all variables in `graph`.

with tf.Session(graph=graph) as sess:
    saver.restore(sess)
    # Do some work with the model....

或者，您可以创建 tf.Session inside with graph.as_default(): 块，在这种情况下它将使用graph 用于其所有操作.

Alternatively, you can create the tf.Session inside the with graph.as_default(): block, in which case it will use graph for all of its operations.

这篇关于在 Tensorflow 中没有保存错误的变量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！