张量流图中的梯度计算错误吗? [英] Is gradient in the tensorflow's graph calculated incorrectly?

问题描述

tensorflow 中一个非常简单的例子:min (x + 1)^2 其中 x 是一个标量.代码是:

A very simple example in tensorflow: min (x + 1)^2 where x is a scalar. The code is:

import tensorflow as tf

x = tf.Variable(initial_value=3.0)
add = tf.add(x, 1)
y = tf.square(add)
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(y)

然后将图形写入磁盘:

graph = tf.get_default_graph()
writer = tf.summary.FileWriter("some/dir/to/write/events")
writer.add_graph(graph=graph)

最后，在张量板中可视化，它看起来像这样:

Finally, visualizing it in tensorboard, it looks like this:

问题是，为什么添加"是与梯度连接的节点?我想因为我试图最小化 y，所以平方"节点应该是.这是一个错误吗?谁能解释一下?

The question is, why is the "Add" node connected with the gradients? I think since I am trying to minimize y, the "Square" node should be. Is this a bug? Can anyone explain it?

推荐答案

不涉及错误.您只需要了解什么是梯度并知道如何自己计算.所以(x+1)^2' = 2*(x+1).这意味着您不需要计算 (x+1)^2 来计算梯度.如果您放大渐变部分，您会看到它计算了正方形的渐变并确定它在那里需要图形的哪一部分:

There is no bug involved. You just need to understand what is a gradient and know how to compute one yourself. So (x+1)^2' = 2*(x+1). Which means that you do not need to calculate (x+1)^2 to calculate the gradient. If you will zoom in the gradient part you will see that it calculated the gradient of your square and figured out that it does which part of the graph is needed there :

这是一个更有趣、更直观的例子:

Here is a more interesting and more intuitive example:

import tensorflow as tf

x = tf.Variable(initial_value=3.0)
y = tf.cos(x)

train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(y)

with tf.Session() as sess:
    writer = tf.summary.FileWriter('logs', sess.graph)
    writer.close()

你应该知道cos(x)' = - sin(x).这意味着只需要 x 来计算梯度.这就是你在图中看到的:

You should know that cos(x)' = - sin(x). Which means that only x is needed to calculate gradient. And this is what you see in the graph:

这篇关于张量流图中的梯度计算错误吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！