在TensorFlow张量上调用Keras模型但保持权重 [英] Calling a Keras model on a TensorFlow tensor but keep weights

问题描述

在 Keras作为TensorFlow的简化接口:教程描述了如何在TensorFlow张量上调用Keras模型.

In Keras as a simplified interface to TensorFlow: tutorial they describe how one can call a Keras model on a TensorFlow tensor.

from keras.models import Sequential

model = Sequential()
model.add(Dense(32, activation='relu', input_dim=784))
model.add(Dense(10, activation='softmax'))

# this works! 
x = tf.placeholder(tf.float32, shape=(None, 784))
y = model(x)

他们还说:

注意:通过调用Keras模型，您正在重用其架构和权重.当在张量上调用模型时，您将在输入张量的顶部创建新的TF op，而这些op将重用模型中已经存在的TF Variable实例.

Note: by calling a Keras model, your are reusing both its architecture and its weights. When you are calling a model on a tensor, you are creating new TF ops on top of the input tensor, and these ops are reusing the TF Variable instances already present in the model.

我将其解释为，在y中，模型的权重将与模型中的权重相同.但是，对我而言，似乎重新初始化了结果Tensorflow节点中的权重.下面是一个最小的示例:

I interpret this as that the weights of the model will be the same in y as in model. However, for me it seems like the weights in the resulting Tensorflow node are reinitialized. A minimal example can be seen below:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
# Create model with weight initialized to 1
model = Sequential()
model.add(Dense(1, input_dim=1, kernel_initializer='ones',
                bias_initializer='zeros'))
model.compile(loss='binary_crossentropy', optimizer='adam',
              metrics=['accuracy'])

# Save the weights 
model.save_weights('file')

# Create another identical model except with weight initialized to 0
model2 = Sequential()
model2.add(Dense(1, input_dim=1, kernel_initializer='zeros',
                 bias_initializer='zeros'))
model2.compile(loss='binary_crossentropy', optimizer='adam',
               metrics=['accuracy'])
# Load the weight from the first model
model2.load_weights('file')
# Call model with Tensorflow tensor
v = tf.Variable([[1, ], ], dtype=tf.float32)
node = model2(v)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
print(sess.run(node), model2.predict(np.array([[1, ], ])))
# Prints (array([[ 0.]], dtype=float32), array([[ 1.]], dtype=float32))

为什么要这样做:

我想在另一个最小化方案中使用经过训练的网络，这是不允许在搜索空间中惩罚"网络的地方.因此，如果您有不涉及此特定方法的想法，也将不胜感激.

Why I want to do this:

I want to use a trained network in another minimization scheme were the network "punish" places in the search space that are not allowed. So if you have ideas not involving this specific approach, that is also very appreciated.

推荐答案

终于找到答案了.问题中的示例存在两个问题.

Finally found the answer. There are two problems in the example from the question.

第一个也是最明显的是，我调用了tf.global_variables_intializer()函数，该函数将重新初始化会话中的所有变量.相反，我应该调用tf.variables_initializer(var_list)，其中var_list是要初始化的变量列表.

The first and most obvious was that I called the tf.global_variables_intializer() function which will re-initialize all variables in the session. Instead I should have called the tf.variables_initializer(var_list) where var_list is a list of variables to initialize.

第二个问题是Keras没有使用与本地Tensorflow对象相同的会话.这意味着要能够在我的会话sess中运行tensorflow对象model2(v)，需要重新初始化.再次将Keras作为tensorflow的简化接口:教程能够提供帮助

The second problem was that Keras did not use the same session as the native Tensorflow objects. This meant that to be able to run the tensorflow object model2(v) with my session sess it needed to be reinitialized. Again Keras as a simplified interface to tensorflow: Tutorial was able to help

我们应该先创建一个TensorFlow会话并在Keras中注册它.这意味着Keras将使用我们注册的会话初始化它在内部创建的所有变量.

We should start by creating a TensorFlow session and registering it with Keras. This means that Keras will use the session we registered to initialize all variables that it creates internally.

import tensorflow as tf
sess = tf.Session()

from keras import backend as K
K.set_session(sess)

如果将这些更改应用于我的问题中提供的示例，我们将获得以下代码，该代码完全可以完成预期的工作.

If we apply these changes to the example provided in my question we get the following code that does exactly what is expected from it.

from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense
sess = tf.Session()
# Register session with Keras
K.set_session(sess)
model = Sequential()
model.add(Dense(1, input_dim=1, kernel_initializer='ones',
                bias_initializer='zeros'))
model.compile(loss='binary_crossentropy', optimizer='adam',
              metrics=['accuracy'])
model.save_weights('test')

model2 = Sequential()
model2.add(Dense(1, input_dim=1, kernel_initializer='zeros',
                 bias_initializer='zeros'))
model2.compile(loss='binary_crossentropy', optimizer='adam',
               metrics=['accuracy'])
model2.load_weights('test')
v = tf.Variable([[1, ], ], dtype=tf.float32)
node = model2(v)
init = tf.variables_initializer([v, ])
sess.run(init)
print(sess.run(node), model2.predict(np.array([[1, ], ])))
# prints: (array([[ 1.]], dtype=float32), array([[ 1.]], dtype=float32))

结论:

教训是，在混合Tensorflow和Keras时，请确保所有内容都使用相同的会话.

Conclusion:

The lesson is that when mixing Tensorflow and Keras, make sure everything uses the same session.

这篇关于在TensorFlow张量上调用Keras模型但保持权重的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！