取Tensorflow中的梯度tf.gradient [英] taking the gradient in Tensorflow, tf.gradient

问题描述

我正在使用这个tensorflow函数来获取我的函数jacobian.遇到两个问题:

I am using this function of tensorflow to get my function jacobian. Came across two problems:

1. 如果我没有记错的话，以下两段中的tensorflow文档与其自身矛盾:

gradients()将ops添加到图中，以输出ys相对于xs的偏导数.它返回一个长度为len(xs)的张量的列表，其中每个张量是ys中y的总和(dy/dx). 引用

gradients() adds ops to the graph to output the partial derivatives of ys with respect to xs. It returns a list of Tensor of length len(xs) where each tensor is the sum(dy/dx) for y in ys. Blockquote

Blockquote 返回值: xs中每个x的sum(dy/dx)列表. 引用

Blockquote Returns: A list of sum(dy/dx) for each x in xs. Blockquote

根据我的测试，实际上是返回len(ys)的向量，该向量是xs中每个x的总和(dy/dx).

According to my test, it is, in fact, return a vector of len(ys) which is the sum(dy/dx) for each x in xs.

1. 我不明白为什么他们以返回值是列(或行，取决于您如何定义Jacobian)的总和的方式设计它.

1. I do not understand why they designed it in a way that the return is the sum of the columns(or row, depending on how you define your Jacobian).

我怎么才能真正得到雅可比式的?

How can I really get the Jacobian?

4.在损失中，我需要函数相对于输入(x)的偏导数，但是当我针对网络权重进行优化时，我将x定义为占位符，其值随后被馈入，并且权重是可变的，在这种情况下，我是否仍可以相对于输入(x)定义函数的符号导数?并把它放在损失中? (稍后当我们针对权重进行优化时，将带来函数的二阶导数.)

4.In the loss, I need the partial derivative of my function with respect to input (x), but when I am optimizing with respect to the network weights, I define x as a placeholder whose value is fed later, and weights are variable, in this case, can I still define the symbolic derivative of function with respect to input (x)? and put it in the loss? ( which later when we optimize with respect to weights will bring second order derivative of the function.)

推荐答案

1. 我认为您是对的，那里有一个错字，可能是长度为len(ys)".

1. I think you are right and there is a typo there, it was probably meant to be "of length len(ys)".

为了提高效率.我无法确切解释其原因，但这似乎是TensorFlow如何处理自动区分的一个非常基本的特征.参见问题#675 .

For efficiency. I can't explain exactly the reasoning, but this seems to be a pretty fundamental characteristic of how TensorFlow handles automatic differentiation. See issue #675.

在TensorFlow中没有直接的方法来获取Jacobian矩阵.看看问题#675 .基本上，您需要在每列/行中调用一次tf.gradients.

There is no straightforward way to get the Jacobian matrix in TensorFlow. Take a look at this answer and again issue #675. Basically, you need one call to tf.gradients per column/row.

是的，当然.您可以计算所需的任何梯度，占位符与任何其他操作之间实际上没有任何实际差异.有些操作没有梯度，因为它的定义不明确或未实现(在这种情况下，通常将返回0)，仅此而已.

Yes, of course. You can compute whatever gradients you want, there is no real difference between a placeholder and any other operation really. There are a few operations that do not have a gradient because it is not well defined or not implemented (in which case it will generally return 0), but that's all.

这篇关于取Tensorflow中的梯度tf.gradient的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！