TensorFlow Dataset 的函数 cache() 和 prefetch() 有什么作用? [英] What do the TensorFlow Dataset's functions cache() and prefetch() do?

问题描述

I am following TensorFlow's Image Segmentation tutorial. In there there are the following lines:

train_dataset = train.cache().shuffle(BUFFER_SIZE).batch(BATCH_SIZE).repeat()
train_dataset = train_dataset.prefetch(buffer_size=tf.data.experimental.AUTOTUNE)

1. What does the cache() function do? The official documentation is pretty obscure and self-referencing:

Caches the elements in this dataset.

2. What does the prefetch() function do? The official documentation is again pretty obscure:

Creates a Dataset that prefetches elements from this dataset.

解决方案

The tf.data.Dataset.cache transformation can cache a dataset, either in memory or on local storage. This will save some operations (like file opening and data reading) from being executed during each epoch. The next epochs will reuse the data cached by the cache transformation.

You can find more about the cache in tensorflow here.

Prefetch overlaps the preprocessing and model execution of a training step. While the model is executing training step s, the input pipeline is reading the data for step s+1. Doing so reduces the step time to the maximum (as opposed to the sum) of the training and the time it takes to extract the data.

You can find more about prefetch in tensorflow here.

Hope this answers your question. Happy Learning.

这篇关于TensorFlow Dataset 的函数 cache() 和 prefetch() 有什么作用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！