如何计算 Tensorflow 中的所有二阶导数(仅 Hessian 矩阵的对角线)? [英] How to compute all second derivatives (only the diagonal of the Hessian matrix) in Tensorflow?

问题描述

我有一个 损失 值/函数，我想计算关于张量 f(大小为 n)的所有二阶导数.我设法使用 tf.gradients 两次，但是当第二次应用它时，它对第一个输入的导数求和(请参阅我的代码中的 second_derivatives).

I have a loss value/function and I would like to compute all the second derivatives with respect to a tensor f (of size n). I managed to use tf.gradients twice, but when applying it for the second time, it sums the derivatives across the first input (see second_derivatives in my code).

我还设法检索了 Hessian 矩阵，但我只想计算它的对角线以避免额外计算.

Also I managed to retrieve the Hessian matrix, but I would like to only compute its diagonal to avoid extra-computation.

import tensorflow as tf
import numpy as np

f = tf.Variable(np.array([[1., 2., 0]]).T)
loss = tf.reduce_prod(f ** 2 - 3 * f + 1)

first_derivatives = tf.gradients(loss, f)[0]

second_derivatives = tf.gradients(first_derivatives, f)[0]

hessian = [tf.gradients(first_derivatives[i,0], f)[0][:,0] for i in range(3)]

model = tf.initialize_all_variables()
with tf.Session() as sess:
    sess.run(model)
    print "\nloss\n", sess.run(loss)
    print "\nloss'\n", sess.run(first_derivatives)
    print "\nloss''\n", sess.run(second_derivatives)
    hessian_value = np.array(map(list, sess.run(hessian)))
    print "\nHessian\n", hessian_value

我的想法是 tf.gradients(first_derivatives, f[0, 0])[0] 可以检索例如关于 f_0 的二阶导数，但似乎 tensorflow 没有t 允许从张量的切片中导出.

My thinking was that tf.gradients(first_derivatives, f[0, 0])[0] would work to retrieve for instance the second derivative with respect to f_0 but it seems that tensorflow doesn't allow to derive from a slice of a tensor.

推荐答案

现在考虑 tf.hessians，

Now consider tf.hessians,

tf.hessians(loss, f)

https://www.tensorflow.org/api_docs/python/tf/hessians

这篇关于如何计算 Tensorflow 中的所有二阶导数(仅 Hessian 矩阵的对角线)?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！