从 Tensorflow 中的 .tfrecords 文件获取记录总数 [英] Obtaining total number of records from .tfrecords file in Tensorflow

问题描述

是否可以从 .tfrecords 文件中获取记录总数?与此相关的是，人们通常如何跟踪训练模型时已经过去的 epoch 数?虽然我们可以指定 batch_size 和 num_of_epochs，但我不确定获取诸如 current epoch 之类的值是否简单，每个时期的批次数等 - 这样我就可以更好地控制训练的进展情况.目前，我只是使用一个肮脏的黑客来计算这个，因为我事先知道我的 .tfrecords 文件中有多少记录以及我的小批量的大小.感谢任何帮助..

Is it possible for obtain the total number of records from a .tfrecords file ? Related to this, how does one generally keep track of the number of epochs that have elapsed while training models? While it is possible for us to specify the batch_size and num_of_epochs, I am not sure if it is straightforward to obtain values such as current epoch, number of batches per epoch etc - just so that I could have more control of how the training is progressing. Currently, I'm just using a dirty hack to compute this as I know before hand how many records there are in my .tfrecords file and the size of my minibatches. Appreciate any help..

推荐答案

要统计记录数，应该可以使用 tf.python_io.tf_record_iterator.

To count the number of records, you should be able to use tf.python_io.tf_record_iterator.

c = 0
for fn in tf_records_filenames:
  for record in tf.python_io.tf_record_iterator(fn):
     c += 1

为了跟踪模型训练，tensorboard 派上用场.

To just keep track of the model training, tensorboard comes in handy.

这篇关于从 Tensorflow 中的 .tfrecords 文件获取记录总数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！