对于自定义 Python 代码，是否有替代 tf.py_function() 的方法? [英] Is there an alternative to tf.py_function() for custom Python code?

问题描述

我已经开始使用 TensorFlow 2.0，并且在某一方面有点不确定.

I have started using TensorFlow 2.0 and have a little uncertainty with regard to one aspect.

假设我有这个用例:在使用 tf.data.Dataset 摄取数据时，我想对一些图像应用一些特定的增强操作.但是，我使用的外部库要求图像是一个 numpy 数组，不是张量.

Suppose I have this use case: while ingesting data with the tf.data.Dataset I want to apply some specific augmentation operations upon some images. However, the external libraries that I am using require that the image is a numpy array, not a tensor.

当使用 tf.data.Dataset.from_tensor_slices() 时，流动数据需要是 Tensor 类型.具体例子:

When using tf.data.Dataset.from_tensor_slices(), the flowing data needs to be of type Tensor. Concrete example:

def my_function(tensor_image):
   print(tensor_image.numpy()
   return


data = tf.data.Dataset.from_tensor_slices(tensor_images).map(my_function)

上面的代码不工作产生一个

The code above does not work yielding an

'Tensor' 对象没有属性 'numpy' 错误.

'Tensor' object has no attribute 'numpy' error.

我已阅读有关 TensorFlow 2.0 的文档，其中指出，如果您想使用任意的 Python 逻辑，则应使用 tf.py_function 或仅使用 TensorFlow 原语，具体如下:如何转换张量"到numpy"张量流中的数组?

I have read the documentation on TensorFlow 2.0 stating that if one wants to use an arbitrary python logic, one should use tf.py_function or only TensorFlow primitives according to: How to convert "tensor" to "numpy" array in tensorflow?

我的问题如下:是否有另一种方法可以在带有自定义装饰器的函数中使用任意 Python 代码/比使用 tf.py_function 更简单的方法?

My question is the following: Is there another way to use arbitrary python code in a function with a custom decorator/an easier way than to use tf.py_function?

老实说，似乎必须有一种比传递给 tf.py_function 更优雅的方法，转换为 numpy 数组，执行操作 A、B、C、D，然后重新转换为张量并产生结果.

To me honestly it seems that there must be a more elegant way than passing to a tf.py_function, transforming to a numpy array, perform operations A,B,C,D and then retransform to a tensor and yield the result.

推荐答案

没有其他方法可以做到，因为 tf.data.Dataset 仍然存在(而且它们将永远存在，我假设，出于性能原因)在图形模式下执行，因此，您不能使用 tf.* 方法之外的任何东西，TensorFlow 可以轻松地将其转换为其图形表示.

There is no other way of doing it, because tf.data.Datasets are still (and they will always be, I suppose, for performance reasons) executed in graph mode and, thus, you cannot use anything outside of the tf.* methods, that can be easily converted by TensorFlow to its graph representation.

使用 tf.py_function 是在使用 tf.data.Dataset 对象(与使用 TensorFlow 2.0 时发生的情况相反，默认情况下，渴望自然地允许这种混合执行).

Using tf.py_function is the only way to mix Python execution (and thus, you can use any Python library) and graph execution when using a tf.data.Dataset object (on the contrary of what happens when using TensorFlow 2.0, that being eager by default allow this mixed execution naturally).

这篇关于对于自定义 Python 代码，是否有替代 tf.py_function() 的方法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！