访问相对于输入的keras模型输出的梯度值 [英] Accessing gradient values of keras model outputs with respect to inputs

问题描述

作为入门练习，我在Keras中为我做了一个非常简单的NN模型，以对我进行一些非线性回归.我将我的jupyter笔记本计算机作为要点上传到此处(在github上适当显示)，它很短，而且到这一点.

I made a pretty simple NN model to do some non-linear regressions for me in Keras, as an introduction exercise. I uploaded my jupyter notebookit as a gist here (renders properly on github), which is pretty short and to the point.

它恰好适合一维函数y =(x-5)^ 2/25.

It just fits the 1D function y = (x - 5)^2 / 25.

我知道Theano和Tensorflow的核心是基于图的派生(渐变)传递框架.利用损失函数相对于权重的梯度进行基于梯度步长的优化是其主要目的.

I know that Theano and Tensorflow are, at their core, graph based derivative (gradient) passing frameworks. And utilizing the gradients of loss functions with respect to weights for gradient step-based optimization are the main purpose of that.

但是我想了解的是，如果我可以访问某种东西，给定一个经过训练的模型，它可以对我来说相对于输出层近似于输入的导数(而不是权重或损失函数).因此，对于这种情况，我希望通过网络的导数图为我确定y'= 2(x-5)/25.0，用于在网络当前训练状态下输入x的指示值.

But what I'm trying to get sense of is if I have access to something that, given a trained model, can approximate derivatives of inputs with respect to the output layer for me (not the weights or loss function). So for this case, I would want y' = 2(x-5)/25.0 estimated via the network's derivative graph for me for an indicated value of the input x, in the network's currently trained state.

我在Keras或Theano/TF后端API中是否有任何选择可以做到这一点，或者我需要以某种方式对权重进行自己的链式裁定(或者添加我自己的不可训练的身份"层或某物)?在我的笔记本中，您可以看到我尝试了一些基于到目前为止我能找到的方法的方法，但是并没有取得很大的成功.

Do I have any options in either the Keras or Theano/TF backend APIs to do this, or do I need to do my own chain ruling somehow with the weights (or maybe adding my own non-trainable "identity" layers or something)? In my notebook, you can see me trying a few approaches based what I was able to find so far, but without a ton of success.

具体来说，我有一个结构正常的keras模型:

To make it concrete, I have a working keras model with the structure:

model = Sequential()
# 1d input
model.add(Dense(64, input_dim=1, activation='relu'))
model.add(Activation("linear"))
model.add(Dense(32, activation='relu'))
model.add(Activation("linear"))
model.add(Dense(32, activation='relu'))
# 1d output
model.add(Dense(1))

model.compile(loss='mse', optimizer='adam', metrics=["accuracy"])
model.fit(x, y,
      batch_size=10,
      epochs=25,
      verbose=0,
      validation_data=(x_test, y_test))

我想估计相对于输入x的输出y的导数，例如x = 0.5.

I would like to estimate the derivative of output y with respect to input x at, say, x = 0.5.

我所有基于搜索过去的答案来提取梯度值的尝试都导致了语法错误.从高级的角度来看，这是Keras的受支持功能，还是任何解决方案都将特定于后端?

All of my attempts to extract gradient values based on searching for past answers have led to syntax errors. From a high level point of view, is this a supported feature of Keras, or is any solution going to be backend-specific?

推荐答案

如前所述，Theano和TF是象征性的，因此进行派生应该很容易:

As you mention, Theano and TF are symbolic, so doing a derivative should be quite easy:

import theano
import theano.tensor as T
import keras.backend as K
J = T.grad(model.output[0, 0], model.input)
jacobian = K.function([model.input, K.learning_phase()], [J])

首先，在给定输入的情况下计算输出的符号梯度(T.grad)，然后构建一个可以调用并进行计算的函数.请注意，有时由于形状问题，这并不是一件容易的事，因为您为输入中的每个元素获得了一个导数.

First you compute the symbolic gradient (T.grad) of the output given the input, then you build a function that you can call and does the computation. Note that sometimes this is not that trivial due to shape problems, as you get one derivative for each element in the input.

这篇关于访问相对于输入的keras模型输出的梯度值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！