使用 tf.contrib.graph_editor 克隆网络 [英] Cloning a network with tf.contrib.graph_editor

问题描述

我有一个像这样构建网络的函数.

I have a function like this that builds a network.

def build_network(inputs):
  # Some arbitrary set of variables and ops here. For example...
  out = tf.contrib.layers.fully_connected(inputs, 123)
  (...)
  return out

然后我用它来构建这样的网络.

Then I use it to build a network like this.

inputs = tf.placeholder(...)
outputs = build_network(inputs)

如果我想构建更多具有相同结构但独立变量的网络，我只需要在其他一些变量范围和可选的其他输入下再次调用 build_network.

If I wanted to build more networks with identical structure but independent variables I'd just have to call build_network again under some other variable scope and, optionally, other inputs.

我的问题是:如果这个 build_network 不再可用，但原始网络的输入和输出可用，我该怎么做?换句话说:我怎样才能将整个子图从 outputs 一直到 inputs 克隆到另一个具有独立变量集但结构相同的变量范围?

My question is: how can I do this if this build_network is no longer available, but the inputs and outputs of the original network are? In other words: how can I clone the entire subgraph from outputs all the way to inputs into another variable scope with its own independent set of variables but identical structure?

我的理解是 tf.contrib.graph_editor 尤其是 graph_editor.copy 正是我做这些事情所需的工具.但是，我找不到任何使用它们的好例子.有什么建议吗?

My understanding is that tf.contrib.graph_editor in general and graph_editor.copy in particular are precisely the tools I need to do these kind of things. However, I couldn't find any good example of their use. Any suggestions?

推荐答案

回应我自己，我找到了一种复制子图的方法.

Responding myself, I have found what looks like a way to copy the subgraph.

from tensorflow.contrib import graph_editor as ge

# From the example above.
inputs = [tf.placeholder(...), ...]
outputs = build_network(inputs)

sgv = ge.make_view(ge.get_within_boundary_ops(
    tf.get_default_graph(),
    [t.op for t in outputs],
    [t.op for t in inputs]))

# This could be any new inputs. In this example I build new identical placeholders.
new_inputs = {p: tf.placeholder(dtype=p.dtype, shape=p.shape) for p in inputs}
new_sgv, info = ge.copy_with_input_replacements(sgv, new_inputs, dst_scope='copy')

new_inputs = [info.transformed(t) for t in inputs]
new_outputs = [info.transformed(t) for t in outputs]

但是，现在我在尝试使用网络副本时遇到了一个新问题.副本中的新变量未初始化，尝试运行 tf.global_variables_initializer() 也无济于事.

However, now I'm facing a new problem when trying to use the network copy. The new variables in the copy are not initialized, and trying to run tf.global_variables_initializer() does not help.

原因是因为这些 tf.Variable 从未构建过，所以它们不是 GlobalKeys.GLOBAL_VARIABLES 集合的一部分.我可以轻松找到与这些变量对应的操作以及它们在原始和副本之间的映射，但我无法从中构建 tf.Variable.

The reason is that since the tf.Variable for these were never built, they are not part of the GlobalKeys.GLOBAL_VARIABLES collection. I can easily find the ops corresponding to these variables and their mapping between original and copy, but I cannot build a tf.Variable from that.

我发现了一些进行初始化的hacky 解决方法，但它仅适用于集合中的变量.

I found bit of a hacky workaround to do the initialization, but it only works for vars in a collection.

init_ops = []
for v in tf.get_collection(tf.GraphKeys.GLOBAL_VARIABLES):
  if v.op in sgv.ops:
    init_ops.append(info.transformed(v.initializer))

...

session.run([tf.global_variables_initializer()] + init_ops)

有没有更好的方法来做到这一点?理想情况下，允许为复制的变量创建 tf.Variables 以将它们添加到全局变量集合中.或者，如果这是不可能的，至少是一种获得初始化操作的可靠方法，而无需找到原始网络的 tf.Variable 对象.

Is there any better way to do this? Ideally one that allows creating tf.Variables for the copied vars to add them to the global vars collection. Or, if that weren't possible, at least a reliable way to get the initializer ops without having to find the tf.Variable objects of the original network.

这篇关于使用 tf.contrib.graph_editor 克隆网络的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！