在优化器期间保持变量不变 [英] Holding variables constant during optimizer

问题描述

我有一个损失张量 L 的 TensorFlow 计算图，它取决于 2 个 tf.Variables，A 和 B.

I have a TensorFlow computational graph for a loss tensor L that depends on 2 tf.Variables, A and B.

我想在保持 B 固定的同时在变量 A 上运行梯度上升(A+=L 与 A 的梯度)，反之亦然 - 在保持 A 固定的同时在 B 上运行梯度上升(B+=L 与 B 的梯度).我该怎么做?

I'd like to run gradient ascent on variable A (A+=gradient of L wrt A) while holding B fixed, and vice versa - running gradient ascent on B (B+=gradient of L wrt B) while holding A fixed. How do I do this?

推荐答案

tf.stop_gradient(tensor) 可能就是你要找的.出于梯度计算的目的，张量将被视为常数.您可以创建两个损失，将不同的部分视为常量.

tf.stop_gradient(tensor) might be what you are looking for. The tensor will be treated as constant for gradient computation purposes. You can create two losses with different parts treated as constants.

另一种选择(通常更好)是创建 2 个优化器，但仅显式优化变量的子集，例如

The other option (and often better) would be to create 2 optimizers but explicitly optimize only subsets of variables, e.g.

train_a = tf.train.GradientDescentOptimizer(0.1).minimize(loss_a, var_list=[A])
train_b = tf.train.GradientDescentOptimizer(0.1).minimize(loss_b, var_list=[B])

并且您可以在它们之间迭代更新.

and you can iterate between them on the updates.

这篇关于在优化器期间保持变量不变的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！