使用输入管道时如何替换 feed_dict? [英] How to replace feed_dict when using an input pipeline?

问题描述

假设您有一个网络，到目前为止已经使用 feed_dict 将数据注入到图形中.每隔几个 epoch，我就会通过将任一数据集中的一批输入到我的图中来评估训练和测试损失.

Suppose you have an network that has worked with feed_dict so far to inject data into a graph. Every few epochs, I evaluated the training and test loss by feeding a batch from either dataset to my graph.

现在，出于性能原因，我决定使用输入管道.看看这个虚拟示例:

Now, for performance reasons, I decided to use an input pipeline. Take a look at this dummy example:

import tensorflow as tf
import numpy as np

dataset_size = 200
batch_size= 5
dimension = 4

# create some training dataset
dataset = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_size,dimension)).
    astype(np.float32))

dataset = dataset.batch(batch_size) # take batches

iterator = dataset.make_initializable_iterator()
x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(iterator.initializer)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

        # I want to print the loss from another dataset (test set) here

打印训练数据的损失没问题，但我如何为另一个数据集做这个?当使用 feed_dict 时，我只是从所说的集合中得到一个批次并为其输入 x 的值.

Printing the loss of the training data is no problem, but how do I do this for another dataset? When using feed_dict, I simply got a batch from said set and fed it a value for x.

推荐答案

您可以为此做几件事.一个简单的选择可能是拥有两个数据集和迭代器并使用 tf.cond 在它们之间切换.然而，更强大的方法是使用直接支持它的迭代器.有关各种迭代器类型的说明，请参阅关于如何创建迭代器的指南.例如，使用可重新初始化的迭代器，你可以有这样的东西:

There are several things you can do for that. One simple option could be something like having two datasets and iterators and use tf.cond to switch between them. However, the more powerful way of doing it is to use an iterator that supports this directly. See the guide on how to create iterators for a description of the various iterator types. For example, using a reinitializable iterator you could have something like this:

import tensorflow as tf
import numpy as np

dataset_size = 200
dataset_test_size = 20
batch_size= 5
dimension = 4

# create some training dataset
dataset = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_size,dimension)).
    astype(np.float32))

dataset = dataset.batch(batch_size) # take batches

# create some test dataset
dataset_test = tf.data.Dataset.\
    from_tensor_slices(np.random.normal(2.0,size=(dataset_test_size,dimension)).
    astype(np.float32))

dataset_test = dataset_test.batch(batch_size) # take batches

iterator = tf.data.Iterator.from_structure(dataset.output_types,
                                           dataset.output_shapes)

dataset_init_op = iterator.make_initializer(dataset)
dataset_test_init_op = iterator.make_initializer(dataset_test)

x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(dataset_init_op)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

    # print test loss
    sess.run(dataset_test_init_op)
    for i in range(dataset_test_size//batch_size):
        loss1 = sess.run(loss)
        print('test step {:d}.  batch loss {:f}.'.format(i,loss1))

您可以使用可馈送迭代器做类似的事情，这取决于您觉得更方便的事情，我想即使使用可初始化的迭代器，例如制作一个布尔数据集，然后您可以使用 tf.cond，虽然这不是一个很自然的方式

You can do something similar with a feedable iterator, depending on what you find more convenient, and I suppose even with an initializable iterator, for example making a boolean dataset that then you map to some data with tf.cond, although that would not be a very natural way to do it.

以下是使用可初始化迭代器的方法，实际上比我最初的想法更简洁，所以也许您实际上更喜欢这个:

Here is how you can do it with an initializable iterator, actually in a cleaner way than what I was initially thinking, so maybe you actually like this more:

import tensorflow as tf
import numpy as np

dataset_size = 200
dataset_test_size = 20
batch_size= 5
dimension = 4

# create data
data = tf.constant(np.random.normal(2.0,size=(dataset_size,dimension)), tf.float32)
data_test = tf.constant(np.random.normal(2.0,size=(dataset_test_size,dimension)), tf.float32)
# choose data
testing = tf.placeholder_with_default(False, ())
current_data = tf.cond(testing, lambda: data_test, lambda: data)
# create dataset
dataset = tf.data.Dataset.from_tensor_slices(current_data)
dataset = dataset.batch(batch_size)
# create iterator
iterator = dataset.make_initializable_iterator()

x = tf.cast(iterator.get_next(),tf.float32)
w = tf.Variable(np.random.normal(size=(1,dimension)).astype(np.float32))

loss_func = lambda x,w: tf.reduce_mean(tf.square(x-w)) # notice that the loss function is a mean!
loss = loss_func(x,w) # this is the loss that will be minimized
train_op = tf.train.GradientDescentOptimizer(0.1).minimize(loss)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    # train one epoch
    sess.run(iterator.initializer)
    for i in range(dataset_size//batch_size):
        # the training step will update the weights based on ONE batch of examples each step
        loss1,_ = sess.run([loss,train_op])
        print('train step {:d}.  batch loss {:f}.'.format(i,loss1))

    # print test loss
    sess.run(iterator.initializer, feed_dict={testing: True})
    for i in range(dataset_test_size//batch_size):
        loss1 = sess.run(loss)
        print('test step {:d}.  batch loss {:f}.'.format(i,loss1))

这篇关于使用输入管道时如何替换 feed_dict?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！