计算LSTM模型的Jacobian矩阵-Python [英] Calculate Jacobian Matrix of LSTM Model - Python

问题描述

我有一个训练有素的LSTM模型，其中包含1个LSTM层和3个密集层.我将其用于一个"预测的序列.我有4个输入变量和1个输出变量.我正在使用最近20个时间步长的值来预测我的输出变量的下一个值.该模型的架构如下所示

 模型= Sequential()model.add(LSTM(单位= 120，激活='relu'，return_sequences = False，input_shape =(train_in.shape [1]，5)))model.add(密集(100，activation ='relu'))model.add(密集(50，激活='relu'))model.add(密集(1)) 

训练输入和训练输出的形状如下所示

  train_in.shape，train_out.shape(((89264，20，5)，(89264，)) 

我想为此模型计算雅可比矩阵.假设Y = f(x1，x2，x3，x4)是上述神经网络的表示形式，其中:Y-训练模型的输出变量，f-代表模型的函数；x1，x2，x3，x4-输入参数.

如何计算雅可比矩阵?请分享您对此的想法.还有任何有价值的参考，如果您知道的话.

谢谢:)

解决方案

我找到了一种相对于输入获取LSTM模型输出的雅可比矩阵的方法.我将其张贴在这里，以便将来对某人有所帮助.请分享是否有更好或更简单的方法

 将numpy导入为np将熊猫作为pd导入将tensorflow导入为tftf.compat.v1.enable_eager_execution()#这将启用必须执行的急切执行.tf.executing_eagerly()#检查是否启用了急切执行.应该给"True"数据= pd.read_excel(文件名或位置")#我的数据位于127549行和5列的数据帧的from中(127549 * 5)a = data [:20] #shape是(20,5)b = data [50:70]#形状为(20,5)A = [a，b]#列出清单A = np.array(A)#转换为数组大小(2,20,5)At = tf.convert_to_tensor(A, np.float32) #转换成张量形状#TensorShape([Dimension(2)，Dimension(20)，Dimension(5)])模型= load_model('EKF-LSTM-1.h5')#加载训练后的模型#我有一个受过训练的模型，如上面的问题所示.#此模型的输出为单个值使用tf.GradientTape(persistent = True，watch_accessed_variables = True)作为磁带:tape.watch(在)y1 = model(At)#根据输入变量定义输出打印(y1，类型(y1)#输出tf.Tensor([[0.04251503]，[0.04634088]]，shape =(2，1)，dtype = float32)< class'tensorflow.python.framework.ops.EagerTensor'>jacobian = tape.jacobian(y1，At)#jacobian输出w.r.t两个输入jacobian.shape 

Outupt

  TensorShape([Dimension(2)，Dimension(1)，Dimension(2)，Dimension(20)，Dimension(5)]) 

在这里，我计算了Jacobian w.r.t 2个输入，每个输入的大小为(20,5).如果您只想计算尺寸为(20,5)的一个输入的w.r.t，请使用此

jacobian=tape.jacobian(y1,At[0]) #jacobian of output w.r.t only 1st input in 'At'jacobian.shape 

输出

  TensorShape([Dimension(1)，Dimension(1)，Dimension(1)，Dimension(20)，Dimension(5)]) 

I have a trained LSTM model with 1 LSTM Layer and 3 Dense layers. I am using it for a sequence to One prediction. I have 4 input variables and 1 output variable. I am using the values of the last 20 timesteps to predict the next value of my output variable. The architecture of the model is shown below

model = Sequential()
model.add(LSTM(units = 120, activation ='relu', return_sequences = False,input_shape = 
(train_in.shape[1],5)))
    
model.add(Dense(100,activation='relu'))
model.add(Dense(50,activation='relu'))
model.add(Dense(1))

The shapes of training input and training output are as shown below

train_in.shape , train_out.shape
((89264, 20, 5), (89264,))

I want to calculate the jacobian matrix for this model. Say, Y = f(x1,x2,x3,x4) is the representation of the above neural network where: Y -- Output variable of the trained model, f -- Is the function representing the Model; x1,x2,x3,x4 --input parameters.

How can I calculate the Jacobian Matrix?? Please share your thoughts on this. Also any valuable references if you know any.

Thank you :)

解决方案

I found a way to get the Jacobian matrix for LSTM model output with respect to the input. I am posting it here so that it might help someone in the future. Please share if there is any better or more simple way to do the same

import numpy as np
import pandas as pd
import tensorflow as tf
tf.compat.v1.enable_eager_execution() #This will enable eager execution which is must.

tf.executing_eagerly() #check if eager execution is enabled or not. Should give "True"

data = pd.read_excel("FileName or Location ")
#My data is in the from of dataframe with 127549 rows and 5 columns(127549*5)

a = data[:20]  #shape is (20,5)
b = data[50:70] # shape is (20,5)
A = [a,b]  # making a list
A = np.array(A) # convert into array size (2,20,5) 

At = tf.convert_to_tensor(A, np.float32) #convert into tensor
At.shape # TensorShape([Dimension(2), Dimension(20), Dimension(5)])

model = load_model('EKF-LSTM-1.h5') # Load the trained model
# I have a trained model which is shown in the question above. 
# Output of this model is a single value

with tf.GradientTape(persistent=True,watch_accessed_variables=True) as tape:

tape.watch(At)
y1 = model(At) #defining your output as a function of input variables
print(y1,type(y1)

#output 
tf.Tensor([[0.04251503],[0.04634088]], shape=(2, 1), dtype=float32) <class 
'tensorflow.python.framework.ops.EagerTensor'>

jacobian=tape.jacobian(y1,At) #jacobian of output w.r.t both inputs
jacobian.shape 

Outupt

TensorShape([Dimension(2), Dimension(1), Dimension(2), Dimension(20), Dimension(5)])

Here I calculated Jacobian w.r.t 2 inputs each of size (20,5). If you want to calculate w.r.t to only one input of size (20,5), then use this

jacobian=tape.jacobian(y1,At[0]) #jacobian of output w.r.t only 1st input in 'At'
jacobian.shape 

Output

TensorShape([Dimension(1), Dimension(1), Dimension(1), Dimension(20), Dimension(5)])

这篇关于计算LSTM模型的Jacobian矩阵-Python的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！