如何获取用于steps_per_epoch的tf.dataset中的样本数? [英] How to get number of samples in a tf.dataset for steps_per_epoch?

问题描述

我很好奇我如何设置tf.keras中的steps_per_epoch适合在tf.dataset上进行训练?由于我需要大量示例来进行计算，所以我想知道如何获得它?

i am curious how I can set the steps_per_epoch in tf.keras fit for training on a tf.dataset?. Since I need the number of examples to calculate it I wonder how I get this?

由于它的类型为tf.data，因此可以假定这样做更容易.如果我将steps_per_epoch设置为无"，我将得到未知".

As it is of type tf.data you could assume assume that this is more easier. If I set steps_per_epoch to None I get "unknown".

为什么使用tf.data会使生活变得如此复杂?

Why using tf.data makes life so complicated?

推荐答案

上一个答案很好，但我想指出两点:

The previous answer is good, yet I would like to point out two matters:

1. 下面的代码有效，不再需要使用实验包.

import tensorflow as tf
dataset = tf.data.Dataset.range(42)
#Still prints 42
print(dataset.cardinality().numpy())

1. 如果使用filter谓词，则基数可能返回值-2，因此未知；如果确实在数据集上使用了过滤谓词，请确保以其他方式计算出数据集的长度(例如，在对数据集应用.from_tensor_slices()之前，pandas数据框的长度.

1. If you use the filter predicate, the cardinality may return value -2, hence unknown; if you do use filter predicates on your dataset, ensure that you have calculated in another manner the length of your dataset( for example length of pandas dataframe before applying .from_tensor_slices() on it.

另一个重要的一点是如何设置参数steps_per_epoch和validation_steps:steps_per_epoch == length_of_training_dataset // batch_size, validation_steps == length_of_validation_dataset // batch_size

Another important point is how to set the parameters steps_per_epoch and validation_steps : steps_per_epoch == length_of_training_dataset // batch_size, validation_steps == length_of_validation_dataset // batch_size

此处提供了完整示例:

A full example is available here : How to use repeat() function when building data in Keras?

这篇关于如何获取用于steps_per_epoch的tf.dataset中的样本数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！