当尝试运行 tf.confusion 矩阵时，它给出了序列结束错误 [英] when trying to run tf.confusion matrix it gives end of sequence error

问题描述

我正在使用新的 tensoflow 输入管道准备我的数据集，这是我的代码:

I am preparing my dataset using new tensoflow input pipeline, here is my code:

train_data = tf.data.Dataset.from_tensor_slices(train_images)
train_labels = tf.data.Dataset.from_tensor_slices(train_labels)
train_set = tf.data.Dataset.zip((train_data,train_labels)).shuffle(500).batch(30)

valid_data = tf.data.Dataset.from_tensor_slices(valid_images)
valid_labels = tf.data.Dataset.from_tensor_slices(valid_labels)
valid_set = tf.data.Dataset.zip((valid_data,valid_labels)).shuffle(200).batch(20)

test_data = tf.data.Dataset.from_tensor_slices(test_images)
test_labels = tf.data.Dataset.from_tensor_slices(test_labels)
test_set = tf.data.Dataset.zip((test_data, test_labels)).shuffle(200).batch(20)

# create general iterator
iterator = tf.data.Iterator.from_structure(train_set.output_types, train_set.output_shapes)
next_element = iterator.get_next()
train_init_op = iterator.make_initializer(train_set)
valid_init_op = iterator.make_initializer(valid_set)
test_init_op  = iterator.make_initializer(test_set)

现在我想在训练后为我的 CNN 模型的验证集创建一个混淆矩阵，这是我尝试做的:

Now I wanted to create a confusion matrix for validation set of my CNN model after training, here is what I try to do:

sess.run(valid_init_op)
valid_img, valid_label = next_element
finalprediction = tf.argmax(train_predict, 1)
actualprediction = tf.argmax(valid_label, 1)
confusion_matrix = tf.confusion_matrix(labels=actualprediction,predictions=finalprediction,
                                       num_classes=num_classes,dtype=tf.int32,name=None, weights=None)
print(sess.run(confusion_matrix, feed_dict={keep_prob: 1.0}))

通过这种方式，它创建了混淆矩阵，但仅适用于一批验证集.为此，我尝试收集列表中的所有验证集批次，然后使用该列表创建混淆矩阵:

In this way it creates confusion matrix but only for one batch of validation set. for that I tried to collect all validation set batches in list and then use the list for creating confusion matrix:

val_label_list = []    
sess.run(valid_init_op)
for i in range(valid_iters):
    while True:
          try:
              elem = sess.run(next_element[1])
              val_label_list.append(elem)
          except tf.errors.OutOfRangeError:
              print("End of append.")
          break
val_label_list = np.array(val_label_list)
val_label_list = val_label_list.reshape(40,2)

现在 val_label_list 包含验证集所有批次的标签，我可以用它来创建混淆矩阵:

and now the val_label_list contain the labels for all batches of my validation set and I can use it to create confusion matrix:

finalprediction = tf.argmax(train_predict, 1)
actualprediction = tf.argmax(val_label_list, 1)
confusion = tf.confusion_matrix(labels=actualprediction,predictions=finalprediction, 
                    num_classes=num_classes, dtype=tf.int32,name="Confusion_Matrix")

但是现在当我想运行混淆矩阵并打印它时:

But now when I want to run the confusion matrix and print it:

print(sess.run(confusion, feed_dict={keep_prob: 1.0}))

它给了我一个错误:

OutOfRangeError: End of sequence
     [[Node: IteratorGetNext_5 = IteratorGetNext[output_shapes=[[?,10,32,32], [?,2]], output_types=[DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](Iterator_5)]]

谁能告诉我如何处理这个错误?或者任何其他解决我最初问题的解决方案?

anyone can tell me how to deal with this error? or any other solution that solve my original problem?

推荐答案

问题与图流执行有关.看看这一行:

The problem is related with the graph flow execution. Look at this line:

print(sess.run(confusion, feed_dict={keep_prob: 1.0}))

您正在运行图表以获取混淆"值.所以所有的依赖节点也会被执行.然后:

You are running the graph for getting the 'confusion' value. So all dependent nodes will be also executed. Then:

finalprediction = tf.argmax(train_predict, 1)
actualprediction = tf.argmax(val_label_list, 1)
confusion = tf.confusion_matrix(...)

我猜您对 train_predict 的调用将尝试从已经完全迭代的训练迭代器中获取一个新元素，然后触发错误.

I guess that your call to train_predict will try to obtain a new element from the training iterator which has been already completely iterated and after this the error is triggered.

您应该直接在循环中计算混淆矩阵并将结果累加到一个变量中.

You should compute the confusion matrix directly in the loop and accumulate the results in a variable.

sess.run(valid_init_op)
confusion_matrix = np.zeros(n_labels,n_labels)
while True:
      try:
          conf_matrix = sess.run(confusion)
          confusion_matrix += conf_matrix
      except tf.errors.OutOfRangeError:
          print("End of append.")
      break

这篇关于当尝试运行 tf.confusion 矩阵时，它给出了序列结束错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！