Tensorflow仅对变量的某些元素进行最小化 [英] Tensorflow minimise with respect to only some elements of a variable

问题描述

是否可以通过仅更改变量的某些元素来使损失函数最小化?换句话说，如果我的变量X的长度为2，如何通过更改X[0]并保持X[1]不变来最小化损失函数?

Is it possible to minimise a loss function by changing only some elements of a variable? In other words, if I have a variable X of length 2, how can I minimise my loss function by changing X[0] and keeping X[1] constant?

希望我尝试过的这段代码将描述我的问题:

Hopefully this code I have attempted will describe my problem:

import tensorflow as tf
import tensorflow.contrib.opt as opt

X = tf.Variable([1.0, 2.0])
X0 = tf.Variable([3.0])

Y = tf.constant([2.0, -3.0])

scatter = tf.scatter_update(X, [0], X0)

with tf.control_dependencies([scatter]):
    loss = tf.reduce_sum(tf.squared_difference(X, Y))

opt = opt.ScipyOptimizerInterface(loss, [X0])

init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    opt.minimize(sess)

    print("X: {}".format(X.eval()))
    print("X0: {}".format(X0.eval()))

输出:

INFO:tensorflow:Optimization terminated with:
  Message: b'CONVERGENCE: NORM_OF_PROJECTED_GRADIENT_<=_PGTOL'
  Objective function value: 26.000000
  Number of iterations: 0
  Number of functions evaluations: 1
X: [3. 2.]
X0: [3.]

我想在其中找到X0 = 2的最佳值，从而找到X = [2, 2]

where I would like to to find the optimal value of X0 = 2 and thus X = [2, 2]

修改

这样做的动机:我想导入训练有素的图形/模型，然后根据我拥有的一些新数据来调整某些变量的各个元素.

Motivation for doing this: I would like to import a trained graph/model and then tweak various elements of some of the variables depending on some new data I have.

推荐答案

您可以使用此技巧将梯度计算限制为一个索引:

You can use this trick to restrict the gradient calculation to one index:

import tensorflow as tf
import tensorflow.contrib.opt as opt

X = tf.Variable([1.0, 2.0])

part_X = tf.scatter_nd([[0]], [X[0]], [2])

X_2 = part_X + tf.stop_gradient(-part_X + X)

Y = tf.constant([2.0, -3.0])

loss = tf.reduce_sum(tf.squared_difference(X_2, Y))

opt = opt.ScipyOptimizerInterface(loss, [X])

init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)
    opt.minimize(sess)

    print("X: {}".format(X.eval()))

part_X变为要在与X形状相同的单热矢量中更改的值.由于part_X - part_X为0，因此part_X + tf.stop_gradient(-part_X + X)与正向传递中的X相同.通过tf.stop_gradient可以防止所有不必要的梯度计算.

part_X becomes the value you want to change in a one-hot vector of the same shape as X. part_X + tf.stop_gradient(-part_X + X) is the same as X in the forward pass, since part_X - part_X is 0. However in the backward pass the tf.stop_gradient prevents all unnecessary gradient calculations.

这篇关于Tensorflow仅对变量的某些元素进行最小化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！