在 tf.gradients 中使用 grads_ys 参数 - TensorFlow [英] Use of grads_ys parameter in tf.gradients - TensorFlow

问题描述

我想了解tf.gradients 中的grad_ys 参数.我已经看到它像真实梯度的乘法器一样使用，但它在定义中并不存在.在数学上，整个表达式会是什么样子?

解决方案

更好的符号说明是

I want to understand the grad_ys paramter in tf.gradients. I've seen it used like a multiplyer of the true gradient but its not crear in the definition. Mathematically how would the whole expression look like?

解决方案

Edit: better clarification of notation is here

ys are summed up to make a single scalar y, and then tf.gradients computes dy/dx where x represents variables from xs

grad_ys represent the "starting" backprop value. They are 1 by default, but a different value can be when you want to chain several tf.gradients calls together -- you can pass in the output of previous tf.gradients call into grad_ys to continue the backprop flow.

For formal definition, look at the chained expression in Reverse Accumulation here: https://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation

The term corresponding to dy/dw3 * dw3/dw2 in TensorFlow is a vector of 1's (think of it as if TensorFlow wraps cost with a dummy identity op). When you specify grad_ys this term is replaced with grad_ys instead of vector of 1s

这篇关于在 tf.gradients 中使用 grads_ys 参数 - TensorFlow的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！