张量流图中的梯度计算不正确吗? [英] Is gradient in the tensorflow's graph calculated incorrectly?

问题描述

张量流中的一个非常简单的示例:min (x + 1)^2其中x是标量.代码是:

A very simple example in tensorflow: min (x + 1)^2 where x is a scalar. The code is:

import tensorflow as tf

x = tf.Variable(initial_value=3.0)
add = tf.add(x, 1)
y = tf.square(add)
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.01)
train = optimizer.minimize(y)

然后将图形写入磁盘

graph = tf.get_default_graph()
writer = tf.summary.FileWriter("some/dir/to/write/events")
writer.add_graph(graph=graph)

最终在张量板上将其可视化，看起来像这样

finally visualize it in tensorboard, it looks like this

问题是，为什么节点"Add"与渐变相连?我认为，由于我尝试将y最小化，因此节点"Square"应该是，这是错误吗?有人可以解释吗?

question is, why node "Add" is connected with gradients? I think since I try to minimize y, node "Square" should be, is it a bug? can anyone explain it?

推荐答案

不涉及任何错误.您只需要了解什么是梯度并知道如何自己计算即可.所以(x+1)^2' = 2*(x+1).这意味着您无需计算(x+1)^2即可计算梯度.如果您要放大渐变部分，您会看到它计算出了正方形的渐变，并且发现那里确实需要图形的哪一部分:

There is no bug involved. You just need to understand what is a gradient and know how to compute one yourself. So (x+1)^2' = 2*(x+1). Which means that you do not need to calculate (x+1)^2 to calculate the gradient. If you will zoom in the gradient part you will see that it calculated the gradient of your square and figured out that it does which part of the graph is needed there :

这是一个更有趣，更直观的示例:

Here is a more interesting and more intuitive example:

import tensorflow as tf

x = tf.Variable(initial_value=3.0)
y = tf.cos(x)

train = tf.train.GradientDescentOptimizer(learning_rate=0.01).minimize(y)

with tf.Session() as sess:
    writer = tf.summary.FileWriter('logs', sess.graph)
    writer.close()

您应该知道cos(x)' = - sin(x).这意味着只需要x即可计算梯度.这就是您在图形中看到的内容:

You should know that cos(x)' = - sin(x). Which means that only x is needed to calculate gradient. And this is what you see in the graph:

这篇关于张量流图中的梯度计算不正确吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！