如何获得与 Tensorflow 中的激活相关的损失梯度 [英] How to get the gradients of loss with respect to activations in Tensorflow

问题描述

在 cifar10 示例中，损失相对于参数的梯度可以计算如下:

In the cifar10 example, the gradients of loss with respect to parameters can be computed as follows:

grads_and_vars = opt.compute_gradients(loss)
for grad, var in grads_and_vars:
    # ...

有没有办法得到关于激活(不是参数)的损失梯度，并在 Tensorboard 中观察它们?

Is there any way to get the gradients of loss with respect to activations (not the parameters)， and watch them in Tensorboard?

推荐答案

您可以使用 tf.gradients() 函数来计算任何标量张量相对于任何其他张量的梯度(假设梯度是为那些之间的所有操作定义的两个张量):

You can use the tf.gradients() function to compute the gradient of any scalar tensor with respect to any other tensor (assuming the gradients are defined for all of the ops between those two tensors):

activations = ...
loss = f(..., activations)  # `loss` is some function of `activations`.

grad_wrt_activations, = tf.gradients(loss, [activation])

在 TensorBoard 中进行可视化通常很棘手，因为 grad_wrt_activation(通常)是一个与 activation 具有相同形状的张量.添加 tf.histogram_summary() op 可能是最简单的可视化方法

Visualizing this in TensorBoard is tricky in general, since grad_wrt_activation is (typically) a tensor with the same shape as activation. Adding a tf.histogram_summary() op is probably the easiest way to visualize

# Adds a histogram of `grad_wrt_activations` to the graph, which will be logged
# with the other summaries, and shown in TensorBoard.
tf.histogram_summary("Activation gradient", grad_wrt_activations)

这篇关于如何获得与 Tensorflow 中的激活相关的损失梯度的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！