在Numpy中预处理Tensorflow张量 [英] Preprocess a Tensorflow tensor in Numpy

问题描述

我已经在Tensorflow中设置了CNN，并在其中使用TFRecordReader读取数据.它工作得很好，但是我想做一些比tf.image函数所提供的更多的预处理和数据扩充.我特别想做一些随机缩放.

I have set up a CNN in Tensorflow where I read my data with a TFRecordReader. It works well but I would like to do some more preprocessing and data augmentation than offered by the tf.image functions. I would specifically like to do some randomized scaling.

是否可以在Numpy中处理Tensorflow张量?还是我需要删除TFRecordReader而不是在Numpy中进行所有预处理并使用feed_dict提要数据?我怀疑在训练图像时feed_dict方法的速度很慢，但是我可能错了吗?

Is it possible to process a Tensorflow tensor in Numpy? Or do I need to drop the TFRecordReader and rather do all my preprocessing in Numpy and feed data using the feed_dict? I suspect that the feed_dict method is slow when training on images, but I might be wrong?

推荐答案

如果您可以创建一个自定义I/O管道，使用一个或多个线程从TensorFlow取回中间结果，则应用任意的Python逻辑，然后将它们输入到用于后续处理的队列.生成的程序会稍微复杂一些，但是我建议您看一下线程和队列HOWTO 有关如何开始使用的信息.

If you could create a custom I/O pipeline that fetches intermediate results back from TensorFlow using one or more threads, applies arbitrary Python logic, and then feeds them into a queue for subsequent processing. The resulting program would be somewhat more complicated, but I suggest you look at the threading and queues HOWTO for information on how to get started.

如果您如果您已经使用TensorFlow ops构建了预处理管道，则添加一些自定义Python代码的最简单方法是使用

If you have already built a preprocessing pipeline using TensorFlow ops, the easiest way to add some custom Python code is to use the tf.py_func() operator, which takes a list of Tensor objects, and a Python function that maps one or more NumPy arrays to one or more NumPy arrays.

例如，假设您有一个这样的管道:

For example, let's say you have a pipeline like this:

reader = tf.TFRecordReader(...)
image_t = tf.image.decode_png(tf.parse_single_example(reader.read(), ...))

...您可以使用tf.py_func()来应用一些自定义NumPy处理，如下所示:

...you could use tf.py_func() to apply some custom NumPy processing as follows:

from scipy import ndimage
def preprocess(array):
  # `array` is a NumPy array containing.
  return ndimage.rotate(array, 45)

image_t = tf.py_func(preprocess, [image_t], [tf.float32])

这篇关于在Numpy中预处理Tensorflow张量的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！