TensorFlow:如何在没有 eval 的情况下使用自定义梯度执行 python 函数? [英] TensorFlow: how to do python function with custom gradients without eval?

问题描述

我正在尝试在 python 中编写一些自定义 TensorFlow 函数(使用 tf.py_func)，我想在 python 中计算结果和梯度.我正在使用 gradient_override_map 技巧(例如来自 https://gist.github.com/harpone/3453185b41d8d985356cbe5e57d67342 和 如何在 Tensorflow 中仅使用 Python 制作自定义激活函数?).

I am trying to write some custom TensorFlow functions in python (using tf.py_func) where I want to calculate both the results and the gradients in python. I'm using the gradient_override_map trick (for example from from https://gist.github.com/harpone/3453185b41d8d985356cbe5e57d67342 and How to make a custom activation function with only Python in Tensorflow?).

然而，虽然前向函数得到一个 numpy 数组作为输入，梯度函数得到 Tensors.这是一个问题，取决于函数何时被调用，因为可能没有默认会话，和/或可能没有包含所有必需值的 feed_dict(例如，在 tf.train 优化器中).

However, while the function in the forward direction gets a numpy array as an input, the function for the gradient gets Tensors. This is a problem, depending on when the function gets called, because there may not be a default session, and/or there may not be a feed_dict with all the required values yet (for example, in a tf.train optimizer).

如何在前向和后向函数都获取(并返回)numpy 数组的情况下执行 py_func?

How do I do a py_func where both the forward and backward functions get (and return) numpy arrays?

示例代码:

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

def sin_func(x):
    return np.sin(x)

def sin_grad_func(op, grad):
    x = op.inputs[0].eval()
    grad = grad.eval() # <--- this is what I'd like to avoid
    output_grad = np.cos(x) * grad
    return tf.convert_to_tensor(output_grad)

def py_func(func, inp, Tout, stateful=True, name=None, grad_func=None):
    grad_name = 'PyFuncGrad_' + str(np.random.randint(0, 1E+8))
    tf.RegisterGradient(grad_name)(grad_func)
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": grad_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)

with tf.Session() as sess:
    np_x = np.linspace(0, np.pi, num=1000, dtype=np.float32)
    x = tf.constant(np_x)
    y = py_func(sin_func,
                [x],
                [tf.float32],
                name='np_sin',
                grad_func=sin_grad_func)
    y = y[0]
    gr = tf.gradients(y, [x])
    tf.global_variables_initializer().run()
    plt.plot(y.eval())
    plt.plot(gr[0].eval())

推荐答案

如果你想在你的渐变函数中包含任意 Python 代码，最简单的解决方案是在里面创建另一个 tf.py_func()sin_grad_func():

If you want to include arbitrary Python code in your gradient function, the easiest solution is to create another tf.py_func() inside sin_grad_func():

def sin_grad_func_impl(x, grad):
    return np.cos(x) * grad

def sin_grad_func(op, grad):
    return tf.py_func(sin_grad_func_impl, [x, grad], grad.dtype)

这篇关于TensorFlow:如何在没有 eval 的情况下使用自定义梯度执行 python 函数?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！