使用 Dataset api tensorflow 即时生成 [英] on the fly generation with Dataset api tensorflow

问题描述

我有一个生成特征和目标张量的函数.例如.

I have a function which produces feature and target tensors. E.g.

x,t = myfunc() ##x,t tensors

如何将其与 TensorFlow 的数据集 API 集成以进行持续训练?理想情况下，我想使用数据集来设置批处理、转换等内容.

How can I integrate this with TensorFlow's dataset API for continuous training? Ideally I would like to use dataset to set things like batch, transformations.

编辑澄清:问题是我不仅想将 x 和 t 放在我的图表中，还想从它们中创建一个数据集，以便我可以使用我为(正常)有限数据集实现的相同数据集处理我可以使用可初始化的迭代器加载到内存中并输入到同一个图中.

Edit for clarification: The problem being I would like to not just put x and t in my graph but make a dataset from them so that I can use the same dataset processing that I have implemented for (normal) finite datasets I can load into memory and feed into the same graph using an initializable iterator.

推荐答案

假设 x 和 t 是 tf.Tensor 对象，并且 my_func() 构建了一个 TensorFlow 图，您可以使用以下方法与 `Dataset.map():

Assuming x and t are tf.Tensor objects, and my_func() builds a TensorFlow graph, you may be able to use the following approach with `Dataset.map():

# Creates an infinite dataset with a dummy value. You can make this finite by
# specifying an explicit number of elements to `repeat()`.
dummy_dataset = tf.data.Dataset.from_tensors(0).repeat(None)

# Evaluates `my_func` once for each element in `dummy_dataset`.
dataset = dummy_dataset.map(lambda _: my_func())

这篇关于使用 Dataset api tensorflow 即时生成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！