什么是“模型变量"?在 Tensorflow 和苗条? [英] What are "model variables" in Tensorflow and slim?

问题描述

来自 https://github.com/tensorflow/tensorflow/树/master/tensorflow/contrib/slim,

TF-Slim 通过定义模型变量进一步区分变量，它们是代表模型参数的变量.模型在学习期间训练或微调变量并加载从评估或推理期间的检查点.例子包括由 slim.fully_connected 或 slim.conv2d 层创建的变量.非模型变量是期间使用的所有其他变量学习或评估，但不是实际执行所必需的推理.例如， global_step 是一个使用 during 的变量学习和评估，但它实际上不是模型的一部分.同样，移动平均变量可能反映模型变量，但移动平均线本身并不是模型变量.

TF-Slim further differentiates variables by defining model variables, which are variables that represent parameters of a model. Model variables are trained or fine-tuned during learning and are loaded from a checkpoint during evaluation or inference. Examples include the variables created by a slim.fully_connected or slim.conv2d layer. Non-model variables are all other variables that are used during learning or evaluation but are not required for actually performing inference. For example, the global_step is a variable using during learning and evaluation but it is not actually part of the model. Similarly, moving average variables might mirror model variables, but the moving averages are not themselves model variables.

来自 https://www.tensorflow.org/versions/master/api_docs/python/state_ops/variable_helper_functions,

tf.model_variables()

tf.model_variables()

返回 MODEL_VARIABLES 集合中的所有变量.

Returns all variables in the MODEL_VARIABLES collection.

Slim 创建了移动均值"变量，作为其批处理规范层的一部分，这些层包含在 MODEL_VARIABLES 集合中.

Yet slim creates "moving mean" variables as part of its batch norm layers that are included in the MODEL_VARIABLES collection.

我至少可以看到模型变量"的可能定义:

I can see at least possible definitions of "model variable":

1. 用于推理.
2. 在训练期间进行微调(无论是通过优化器还是其他一些方法，例如移动平均).
3. 存储在检查点

Tensorflow 的模型变量"是由条件 2 定义的，而 slim 的模型变量"是由条件 1 定义的吗?

Is it the case that Tensorflow's "model variables" are defined by condition 2, while slim's "model variables" are defined by condition 1?

推荐答案

简单地说，slim 使用 contrib 层.contrib 层使用 layer_variable_getter(实际上是 _model_variable_getter)来生成model_variable"，这些变量是添加到 [tf.GraphKeys.GLOBAL_VARIABLES, tf.GraphKeys.MODEL_VARIABLES] 集合中的变量.

Simply put, slim uses contrib layers. contrib layers use layer_variable_getter, which is actually _model_variable_getter, to generate "model_variable" that are variables added into [tf.GraphKeys.GLOBAL_VARIABLES, tf.GraphKeys.MODEL_VARIABLES] collections.

因此模型变量是普通变量加上被添加到 MODEL_VARIABLES 集合.contrib 层的变量是模型变量，并在需要时从bias"重命名为biases"，从kernel"重命名为weights".

Therefore model variables are common variables plus being added to MODEL_VARIABLES collection. contrib layers' variables are model variables plus being renamed from 'bias' to 'biases' and from 'kernel' to 'weights' if needed.

这篇关于什么是“模型变量"?在 Tensorflow 和苗条?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！