TensorFlow FIFOQueue 不是 FIFO? [英] TensorFlow FIFOQueue not FIFO?

问题描述

我正在以特定顺序将项目排入 TensorFlow FIFOQueue 并希望能够以相同的顺序将它们出列，但这不是我观察到的行为.

I am enqueuing items in a particular order onto a TensorFlow FIFOQueue and expecting to be able to dequeue them in the same order but this is not the behaviour I am observing.

运行以下独立代码演示了该方法和行为.这已在 Python 2.7(但可能适用于 Python 3)和 TensorFlow 1.1 上运行.

Running the following stand-alone code demonstrates the approach and behaviour. This has been run on Python 2.7 (but might work in Python 3) with TensorFlow 1.1.

from __future__ import division, print_function, unicode_literals
import math
import numpy
import tensorflow as tf
from tensorflow.python.training import queue_runner
from tensorflow.python.ops import control_flow_ops

row_count, column_count = 7, 5
batch_size, step_size = 3, 2

# Create some random data
data = numpy.arange(row_count * column_count).reshape(
    (row_count, column_count))
print(data)

batch_count = int(math.ceil(row_count / batch_size))
step_count = int(math.ceil(column_count / step_size))
print(batch_count, step_count)

slices = tf.train.slice_input_producer([data], num_epochs=1, shuffle=False)
batch = tf.train.batch(slices, batch_size, allow_smaller_final_batch=True)

queue = tf.FIFOQueue(32, dtypes=[batch.dtype])
enqueue_ops = []
dependency = None

for step_index in range(step_count):
    step = tf.strided_slice(
        batch, [0, step_index * step_size],
        [tf.shape(batch)[0], (step_index + 1) * step_size])

    if dependency is None:
        dependency = step
    else:
        step = control_flow_ops.with_dependencies([dependency], step)

    enqueue_ops.append(queue.enqueue(step))

queue_runner.add_queue_runner(queue_runner.QueueRunner(
    queue=queue, enqueue_ops=[tf.group(*enqueue_ops)]))
step = queue.dequeue()

supervisor = tf.train.Supervisor()

with supervisor.managed_session() as session:
    for batch_index in range(batch_count):
        for step_index in range(step_count):
            print("Batch %d, step %d" % (batch_index, step_index))
            print(session.run(step))

预期输出是

Batch 0, step 0
[[ 0  1]
 [ 5  6]
 [10 11]]
Batch 0, step 1
[[ 2  3]
 [ 7  8]
 [12 13]]
Batch 0, step 2
[[ 4]
 [ 9]
 [14]]
Batch 1, step 0
[[15 16]
 [20 21]
 [25 26]]
Batch 1, step 1
[[17 18]
 [22 23]
 [27 28]]
Batch 1, step 2
[[19]
 [24]
 [29]]
Batch 2, step 0
[[30 31]]
Batch 2, step 1
[[32 33]]
Batch 2, step 2
[[34]]

实际输出为

Batch 0, step 0
[[ 0  1]
 [ 5  6]
 [10 11]]
Batch 0, step 1
[[ 4]
 [ 9]
 [14]]
Batch 0, step 2
[[ 2  3]
 [ 7  8]
 [12 13]]
Batch 1, step 0
[[15 16]
 [20 21]
 [25 26]]
Batch 1, step 1
[[19]
 [24]
 [29]]
Batch 1, step 2
[[17 18]
 [22 23]
 [27 28]]
Batch 2, step 0
[[30 31]]
Batch 2, step 1
[[32 33]]
Batch 2, step 2
[[34]]

注意批次 0 和批次 1 中的步骤顺序不正确.我一直无法确定这些步骤的顺序.看起来批次总是有序的，但每个批次中的步骤以随机"顺序出现:它看起来是确定性的，但不是 FIFO.

Note that the order of the steps in batches 0 and 1 are incorrect. I have been unable to determine what the order of the steps is. It appears the batches are always in order but the steps within each batch come out in "random" order: it appears deterministic, but not FIFO.

我尝试过使用和不使用上面代码中使用的显式依赖项声明.我尝试将队列容量设置为 1.我尝试设置 enqueue_ops=enqueue_ops 而不是使用 tf.group 但这些更改都没有帮助，最后一个导致非常奇怪输出.

I have tried with and without the explicit dependency declarations as used in the code above. I have tried setting the queue capacity to 1. I have tried setting enqueue_ops=enqueue_ops instead of using tf.group but none of these changes helped and the final one caused very strange output.

也许 tf.group 以某种方式忽略了依赖?

Maybe tf.group somehow ignores dependencies?

推荐答案

看起来 tensorflow.python.ops.control_flow_ops.with_dependencies 没有像我想象的那样工作，或者我没有正确使用它.如果我改用 tf.control_dependencies 代替，我会得到我需要的行为:

It appears tensorflow.python.ops.control_flow_ops.with_dependencies isn't working as I thought, or I was using it incorrectly. If I switch to to using tf.control_dependencies instead, I get the behaviour I need:

from __future__ import division, print_function, unicode_literals
import math
import numpy
import tensorflow as tf
from tensorflow.python.training import queue_runner

row_count, column_count = 7, 5
batch_size, step_size = 3, 2

# Create some random data
data = numpy.arange(row_count * column_count).reshape(
    (row_count, column_count))
print(data)

batch_count = int(math.ceil(row_count / batch_size))
step_count = int(math.ceil(column_count / step_size))
print(batch_count, step_count)

slices = tf.train.slice_input_producer([data], num_epochs=1, shuffle=False)
batch = tf.train.batch(slices, batch_size, allow_smaller_final_batch=True)

queue = tf.FIFOQueue(32, dtypes=[batch.dtype])
enqueue_ops = []
dependency = None

for step_index in range(step_count):
    step = tf.strided_slice(
        batch, [0, step_index * step_size],
        [tf.shape(batch)[0], (step_index + 1) * step_size])

    if dependency is None:
        dependency = queue.enqueue(step)
    else:
        with tf.control_dependencies([dependency]):
            step = queue.enqueue(step)
            dependency = step

    enqueue_ops.append(step)

queue_runner.add_queue_runner(queue_runner.QueueRunner(
    queue=queue, enqueue_ops=[tf.group(*enqueue_ops)]))
step = queue.dequeue()

supervisor = tf.train.Supervisor()

with supervisor.managed_session() as session:
    for batch_index in range(batch_count):
        for step_index in range(step_count):
            print("Batch %d, step %d" % (batch_index, step_index))
            print(session.run(step))

这个答案的动机是对另一个 SO 问题的回答.

这篇关于TensorFlow FIFOQueue 不是 FIFO?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！