强制依赖变量更新 [英] Forcing dependence on a variable update

问题描述

假设我有一些函数 f 的变量 x:

Say I have some function f of variable x:

x = tf.Variable(1.0)
fx = x*x

和一个更新 x 的操作:

and an op which updates x:

new_x = x.assign(2.0)

并且我想获得由更新的 x 产生的 f 的值.我还以为

and I want to get the value of f resulting from the updated x. I had thought that

with tf.control_dependencies([new_x,]):
    new_fx = tf.identity(fx)    

会强制 new_fx 依赖于更新 new_x，但情况似乎并非如此:

would force new_fx to depend on the update new_x, but this doesn't seem to be the case:

init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)

# prints 1.0, expected 4.0
print "new fx", sess.run(new_fx)

是否有其他方法可以定义 fx 的更新值?

Is there some other way to define the updated value of fx?

显然我可以通过编写类似 new_fx = new_x * new_x 的东西来创建一个新的独立副本，但这会增加图形的大小，并且还需要访问 fx，我更愿意将其视为黑匣子.

Obviously I could create a new independent copy by writing something like new_fx = new_x * new_x but this blows up the graph size, and also requires access to the definition of fx, which I'd prefer to treat as a black box.

编辑:为了激发这一点，下面是我想编写的代码草图:

Edit: to motivate this, here's a sketch of the code I want to write:

# Hamiltonian Monte Carlo update, simplified
def hmc_step(x, momentum, logpdf, n_steps=50): 
    # x and momentum are Variables
    # logpdf is a Tensor with potentially complicated dependence on x

    grad = tf.gradients(logpdf, x)[0]

    # initial position        
    new_x = x

    for i in range(n_steps):
        # update position
        new_x = x.assign(new_x + momentum)

        # update momentum using gradient at *current* position
        with tf.control_dependencies([new_x]):
             momentum = momentum + grad # DOESN'T WORK

        # DOES WORK BUT IS UGLY
        # new_logpdf = define_logpdf(new_x)
        # new_grad = tf.gradients(new_logpdf, new_x)[0]
        # momentum = momentum + new_grad

    # (do some stuff to accept/reject the new x)
    # ....

    return new_x

每次通过循环定义一个新的logpdf副本并重新导出梯度感觉真的很不优雅:它需要访问define_logpdf()并将图形大小放大50倍.有没有更好的方法来做到这一点(除了一些等效的 theano.scan)?

It feels really inelegant to define a new copy of logpdf and rederive gradients each time through the loop: it requires access to define_logpdf() and blows up the graph size by a factor of 50. Is there no better way to do this (barring some equivalent of theano.scan)?

推荐答案

with tf.control_dependencies([op]) 块强制对 op 的控制依赖于其他操作在 with 块中创建.在您的情况下， x*x 是在外部创建的，而 tf.identity 只是获取旧值.这是你想要的:

The with tf.control_dependencies([op]) block enforces control dependency on op to other ops created within with block. In your case, x*x is created outside, and the tf.identity just gets the old value. Here is what you want :

with tf.control_dependencies([new_x,]):
  new_fx = x*x

这篇关于强制依赖变量更新的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！