tensorflow:使用队列运行器有效地提供评估/训练数据 [英] tensorflow: efficient feeding of eval/train data using queue runners

问题描述

我正在尝试运行张量流图来训练模型并使用单独的评估数据集进行定期评估.训练和评估数据都是使用队列运行器实现的.

I'm trying to run a tensorflow graph to train a model and periodically evaluate using a separate evaluation dataset. Both training and evaluation data is implemented using queue runners.

我当前的解决方案是在同一个图中创建两个输入，并使用依赖于 is_training 占位符的 tf.cond.以下代码突出显示了我的问题:

My current solution is to create both inputs in the same graph and use a tf.cond dependent on an is_training placeholder. My issue is highlighted by the following code:

import tensorflow as tf
from tensorflow.models.image.cifar10 import cifar10
from time import time


def get_train_inputs(is_training):
    return cifar10.inputs(False)


def get_eval_inputs(is_training):
    return cifar10.inputs(True)


def get_mixed_inputs(is_training):
    train_inputs = get_train_inputs(None)
    eval_inputs = get_eval_inputs(None)

    return tf.cond(is_training, lambda: train_inputs, lambda: eval_inputs)


def time_inputs(inputs_fn, n_runs=10):
    graph = tf.Graph()
    with graph.as_default():
        is_training = tf.placeholder(dtype=tf.bool, shape=(),
                                     name='is_training')
        images, labels = inputs_fn(is_training)

    with tf.Session(graph=graph) as sess:
        coordinator = tf.train.Coordinator()
        threads = tf.train.start_queue_runners(sess=sess, coord=coordinator)
        t = time()
        for i in range(n_runs):
            im, l = sess.run([images, labels], feed_dict={is_training: True})
        dt = time() - t
        coordinator.request_stop()
        coordinator.join(threads)

    return dt / n_runs

print('Train inputs: %.3f' % time_inputs(get_train_inputs))
print('Eval inputs: %.3f' % time_inputs(get_eval_inputs))
print('Mixed inputs: %.3f' % time_inputs(get_mixed_inputs))

我还必须注释掉 tensorflow/models/image/cifar10/cifar10_inputs.py 的 image_summary 行 133.

I also had to comment out the image_summary line 133 of tensorflow/models/image/cifar10/cifar10_inputs.py.

这产生了以下结果:

Train inputs: 0.055
Eval inputs: 0.050
Mixed inputs: 0.105

在混合情况下，两个输入似乎都被读取/解析，即使只使用了 1 个.有没有办法避免这种冗余计算?或者是否有更好的方法在仍然利用队列运行器设置的训练/评估数据之间切换?

It would seem in the mixed case both inputs are being read/parsed, even though only 1 is used. Is there a way of avoiding this redundant computation? Or is there a nicer way of switching between training/evaluation data that still leverages the queue-runner setup?

推荐答案

你读过这篇link 关于多输入?我认为您可以在输入函数中添加一个 is_training 参数来区分训练数据和评估数据.然后你可以重用共享变量来获取 eval 数据的 logits 并为 eval 构建一个操作.然后在您的图表中，运行 valudation_accuracy=sess.run(eval_op) 以获得 eval 准确性.

Have you read the last section of this link about multi inputs? I think you can add a is_training argument to your input function to distinguish training data from eval data. Then you can reuse sharing variables to get the logits for eval data and build a op for eval. Then in your graph, run valudation_accuracy=sess.run(eval_op) to get eval accuracy.

更新:

据我所知，如果你想训练 n 个批次，评估，训练，评估，你可以在同一个图中保留两个操作，不需要构建一个新的.假设你已经构建了所有需要的函数，那么代码应该是这样的:

Hi, from my understanding,if you want to train for n batches, evaluate, train, evaluate, you can keep there two ops in the same graph, no need to build a new one. Assume you have already build all the needed function, then the code should like this:

#the following two steps will add train and eval input queue to the graph
train_inputs,train_labels = inputs(is_train=True)
eval_inputs,eval_labels = inputs(is_train=False)

with tf.variable_scope("inference") as scope:
    train_logits = inference(train_inputs)
    scope.reuse_variables()
    eval_logits = inference(eval_inputs)

loss = loss(train_logits,train_labels)
eval_accuracy = accuracy(eval_logits,eval_labels)

#...add train op here,start queue runner and train it ...

这篇关于tensorflow:使用队列运行器有效地提供评估/训练数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！