如何在 Tensorflow 队列中以张量属性(“过采样")为条件复制输入张量? [英] How to duplicate input tensors conditional on a tensor attribute ("oversampling") in a Tensorflow queue?

问题描述

我正在为自己的目的移植 Tensorflow Cifar-10 教程文件，但遇到了一个有趣的问题，由于 Tensorflow 的图形和会话架构，我无法轻松地将其概念化.

I am porting the Tensorflow Cifar-10 tutorial files for my own purpose, and have run into an interesting problem that I can not easily conceptualize due to the graph and session architecture of Tensorflow.

问题是我的输入数据集高度不平衡，因此我需要根据标签对输入管道中的某些图像进行过采样"(和增强).在普通的 Python 环境中，我可以设置一个简单的控制流语句，形式为 if label then duplicate，但是由于控制流操作存在于外部，我无法在 Tensorflow 中编写相同的语法在这种情况下，正在运行的会话和 label 不会返回值.

The issue is that my input dataset is highly imbalanced, and as such I need to "oversample" (and augment) certain images in the input pipeline conditional on their labels. In a normal Python environmental I could set up a simple control flow statement of the form if label then duplicate, but I am not able to write the same syntax in Tensorflow due to the control flow operation existing outside of a running session and label in this case does not return a value.

我的问题是，在 Tensorflow 队列中对张量进行过采样的最简单方法是什么?

My question is, what is the easiest method for oversampling a tensor inside of a Tensorflow queue?

我知道我可以在输入操作之前简单地复制感兴趣的数据，但这显然消除了运行时过采样所带来的任何存储节省.

I know that I could simply duplicate the data of interest prior to the input operation, but this obviously removes any storage savings incurred by oversampling during runtime.

我想要做的是评估张量的标签(在 Cifar-10 的情况下，通过检查 1D image.label 属性)，然后通过固定因子复制该张量(例如，如果标签是狗"，则为 4 倍) 并将所有张量发送到批处理操作.我最初的方法是在 Reader 操作之后和批处理操作之前尝试复制步骤，但这也在运行会话之外.我正在考虑使用 TF 的 while 控制流语句，但我不确定这个函数是否有能力做除了修改输入张量之外的任何事情.你怎么看?

What I want to do is evaluate a Tensor's label (in the Cifar-10 case, by checking the 1D image.label attribute) and then duplicate that Tensor by a fixed factor (say, 4x if the label is "dog") and send all the Tensors down to the batching operation. My original approach was to attempt the duplication step after the Reader operation and before the batching operation, but this too is outside of a running session. I was thinking of utilizing TF's while control flow statement but I'm not sure this will function has the ability to do anything other than modify the input Tensor. What do you think?

更新 #1

基本上，我尝试创建一个 py_func() 接收扁平图像字节和标签字节，并根据标签的值垂直堆叠相同的图像字节 N 次，然后将其作为 (N x image_bytes) 张量(py_func() 自动将输入张量转换为 numpy 并返回).我试图从可变高度张量创建一个 input_queue，其形状报告为 (?,image_bytes)，然后实例化一个读取器以获取 image_byte 大小的记录.好吧，似乎您无法根据未知的数据大小构建队列，因此这种方法对我不起作用，事后看来很有意义，但我仍然无法概念化识别队列中记录并重复该记录的方法具体次数.

Basically I attempted to create a py_func() that took in the flattened image bytes and the label bytes, and vertically stack the same image bytes N times depending on the value of the label and then return that as a (N x image_bytes) tensor (py_func() auto converted the input tensor to numpy and back). i attempted to create an input_queue from the variable-height tensor whose shape reports as (?,image_bytes) and then instantiate a reader to rip off image_byte size records. Well it seems like you can't build queues off of unknown data sizes so this approach is not working for me which makes sense in hindsight but I still can't conceptualize a method for identifying a record in a queue, and repeating that record a specific number of times.

更新#2

48 小时后，我终于找到了解决方法，这要归功于 这个 SO 线程 我能够挖掘出来.不过，该线程中概述的解决方案仅假设有 2 类数据，因此如果 pred 为 True，则 tf.cond() 函数足以对一类进行过采样，并进行过采样另一个如果 pred 是 False.为了有一个 n 路条件，我试图建立一个 tf.case() 函数，导致 ValueError: 无法推断 Tensor's rank.结果证明 tf.case() 函数不保留 shape 属性并且图构建失败，因为输入管道末尾的任何批处理操作都必须采用 shape 参数，或根据 文档:

Well after 48 hours I finally figured out a workaround, thanks to this SO thread that I was able to dig up. The solution outlined in that thread only assumes 2 classes of data though, so the tf.cond() function suffices to oversample one class if the pred is True, and to oversample the other if pred is False. In order to have an n-way conditional, I attempted to institute a tf.case() function that resulted in ValueError: Cannot infer Tensor's rank. Turns out that the tf.case() function does not retain shape properties and the graph construction fails as any batching op at the end of the input pipeline must take a shape argument, or take tensors of defined shape, as per this note in the documentation:

注意:您必须确保 (i) 传递了 shape 参数，或 (ii) 张量中的所有张量都必须具有完全定义的形状.如果这两个条件都不成立，就会引发 ValueError.

N.B.: You must ensure that either (i) the shapes argument is passed, or (ii) all of the tensors in tensors must have fully-defined shapes. ValueError will be raised if neither of these conditions holds.

进一步挖掘表明这是一个已知问题，tf.case() 截至 2016 年 12 月尚未解决.这只是 Tensorflow 中众多控制流问题之一.无论如何，我对 n 路过采样问题的精简解决方案是这样的:

Further digging shows that this is a known issue with tf.case() that has yet to be resolved as of December 2016. Just one of the many control-flow head-scratchers in Tensorflow. Anyway, my stripped-down solution to the n-way oversampling issue is thus:

# Initiate a queue of "raw" input data with embedded Queue Runner.
queue = tf.train.string_input_producer(rawdata_filename) 

# Instantiate Reader Op to read examples from files in the filename queue.
reader = tf.FixedLengthRecordReader(record_bytes)

# Pull off one instance, decode and cast image and label to 3D, 1D Tensors.
result.key, value = reader.read(queue)
image_raw, label_raw = decode(value)
image = tf.cast(image_raw, dtype) #3D tensor
label = tf.cast(label_raw, dtype) #1D tensor

# Assume your oversampling factors per class are fixed
# and you have 4 classes.
OVERSAMPLE_FACTOR = [1,2,4,10]

# Now we need to reshape input image tensors to 4D, where the 
# first dimension is the image number in a batch of oversampled tensors.
# images = tf.expand_dims(image, 0) # so, (*,height,width,channels) in 4D

# Set up your predicates, which are 1D boolean tensors.
# Note you will have to squash the boolean tensors to 0-dimension.
# This seems illogical to me, but it is what it is.
pred0 = tf.reshape(tf.equal(label, tf.convert_to_tensor([0])), []) #0D tf.bool
pred1 = tf.reshape(tf.equal(label, tf.convert_to_tensor([1])), []) #0D tf.bool
pred2 = tf.reshape(tf.equal(label, tf.convert_to_tensor([2])), []) #0D tf.bool
pred3 = tf.reshape(tf.equal(label, tf.convert_to_tensor([3])), []) #0D tf.bool

# Build your callables (functions) that vertically stack an input image and
# label tensors X times depending on the accompanying oversample factor.
def f0(): return tf.concat(0, [images]*OVERSAMPLE_FACTOR[0]), tf.concat(0, [label]*OVERSAMPLE_FACTOR[0])
def f1(): return tf.concat(0, [images]*OVERSAMPLE_FACTOR[1]), tf.concat(0, [label]*OVERSAMPLE_FACTOR[1])
def f2(): return tf.concat(0, [images]*OVERSAMPLE_FACTOR[2]), tf.concat(0, [label]*OVERSAMPLE_FACTOR[2])
def f3(): return tf.concat(0, [images]*OVERSAMPLE_FACTOR[3]), tf.concat(0, [label]*OVERSAMPLE_FACTOR[3])

# Here we have N conditionals, one for each class.  These are exclusive
# but due to tf.case() not behaving every conditional gets evaluated.
[images, label] = tf.cond(pred0, f0, lambda: [images,label])
[images, label] = tf.cond(pred1, f1, lambda: [images,label])
[images, label] = tf.cond(pred2, f2, lambda: [images,label])
[images, label] = tf.cond(pred3, f3, lambda: [images,label])

# Pass the 4D batch of oversampled tensors to a batching op at the end
# of the input data queue.  The batching op must be set up to accept
# batches of tensors (4D) as opposed to individual tensors (in our case, 3D).
images, label_batch = tf.train.batch([images, label],
                                     batch_size=batch_size,
                                     num_threads=num_threads,
                                     capacity=capacity,
                                     enqueue_many = True) #accept batches

推荐答案

我的问题的解决方案是一种解决方法，并在原始问题的更新 2"中进行了概述.

The solution to my problem is a workaround, and is outlined in 'Update 2' in the original question.

这篇关于如何在 Tensorflow 队列中以张量属性(“过采样")为条件复制输入张量?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！