在分布式张量流中仅恢复零件模型的推荐方法是什么 [英] What's the recommend way of restoring only parts model in distributed tensorflow

问题描述

当我们在不同的任务上对模型进行微调时，模型中只有一部分变量从预训练任务中恢复，其他变量作为初始值保留.

When we finetune a model on a different task, only a part of vars in the model are restored from the pretrained task and others are left as initial values.

尽可能多的文档推荐(page1 page2)，当用局部图训练时，恢复预训练的运行全局初始化操作后的模型(如果包含 MonitoredSession 或 supervisor，则在init_fn"中调用恢复).

As many docs recommends(page1 page2), when training with a local graph, restoring the pretrained model after running the global init op(call restoring in "init_fn" if MonitoredSession or supervisor is included).

但在分布式情况下，全局 init op ma​​ke "model_ready" 是否在调用恢复模型之前返回 true?其他非主节点将使用未准备好"值.

But in the distributed case, does global init op make "model_ready" returns true before the restoring-model called? other non-chief nodes will use the "not ready" values.

推荐答案

弄清楚.global_variables_initializer 在 facet variable_initializers(global_variables()) 中.所以我们可以只初始化一些选定的变量并从预训练模型中恢复左边.model_ready"将保持为 False，直到所有变量恢复.

Figure it out. global_variables_initializer is in facet variable_initializers(global_variables()). So we can initialize only some selected vars and restore the lefts from pretrained models. "model_ready" will keep as False until all vars are restored.

这篇关于在分布式张量流中仅恢复零件模型的推荐方法是什么的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！