Tensorflow 0.12 版中 tf.Variable.ref() 的替代方案是什么? [英] What is the alternative of tf.Variable.ref() in Tensorflow version 0.12?

问题描述

我正在尝试运行 A3C 强化学习算法的开放代码来学习 A3C 代码

I'm trying to run open code of A3C reinforcement learning algorithm to learn A3C in A3C code

但是，我遇到了几个错误，除了一个我可以修复.代码中使用了tf.Variable的成员函数ref()(1,2)，但在最近的 tensorflow 版本 0.12rc 中，该功能似乎已被弃用.所以我不知道替换它的最佳方法是什么(我不明白作者为什么使用 ref()).当我刚刚将其更改为变量本身时(例如将 v.ref() 更改为 v)，没有错误，但没有更改奖励.似乎无法学习，我猜是因为变量没有正确更新.

However,I got several errors and I could fix except one. In the code, ref() which is a member function of tf.Variable is used (1,2), but in recent tensorflow version 0.12rc, that function seems to be deprecated. So I don't know what is the best way to replace it (I don't understand exactly why the author used ref()). When I just changed it to the variable itself (for example v.ref() to v), there was no error, but reward is not changed. It seems it cannot learn and I guess it is because the variables are not properly updated.

请告诉我修改代码的正确方法是什么.

Please advise me what is the proper way to modify the code to work.

推荐答案

新方法 tf.Variable.read_value() 是 TensorFlow 0.12 及更高版本中 tf.Variable.ref() 的替代品.

此方法的用例有点难以解释，其动机是某些缓存行为导致在不同设备上多次使用远程变量以使用缓存值.假设您有以下代码:

The use case for this method is slightly tricky to explain, and is motivated by some caching behavior that causes multiple uses of a remote variable on a different device to use a cached value. Let's say you have the following code:

with tf.device("/cpu:0")
  v = tf.Variable([[1.]])

with tf.device("/gpu:0")
  # The value of `v` will be captured at this point and cached until `m2`
  # is computed.
  m1 = tf.matmul(v, ...)

with tf.control_dependencies([m1])
  # The assign happens (on the GPU) after `m1`, but before `m2` is computed.
  assign_op = v.assign([[2.]])

with tf.control_dependencies([assign_op]):
  with tf.device("/gpu:0"):
    # The initially read value of `v` (i.e. [[1.]]) will be used here,
    # even though `m2` is computed after the assign.
    m2 = tf.matmul(v, ...)

sess.run(m2)

您可以使用 tf.Variable.read_value() 强制 TensorFlow 稍后再次读取该变量，并且它将受到任何控制依赖项的影响.所以如果你想在计算 m2 时看到赋值的结果，你可以修改程序的最后一个块，如下所示:

You can use tf.Variable.read_value() to force TensorFlow to read the variable again later, and it will be subject to whatever control dependencies are in place. So if you wanted to see the result of the assign when computing m2, you'd modify the last block of the program as follows:

with tf.control_dependencies([assign_op]):
  with tf.device("/gpu:0"):
    # The `read_value()` call will cause TensorFlow to transfer the
    # new value of `v` from the CPU to the GPU before computing `m2`.
    m2 = tf.matmul(v.read_value(), ...)

(请注意，目前，如果所有操作都在同一台设备上，则您不需要使用 read_value()，因为 TensorFlow 不会当变量被用作同一设备上的操作的输入时，制作该变量的副本.这可能会导致很多混乱——例如，当您将变量排入队列时！—这是我们正在努力增强变量的内存模型.)

(Note that, currently, if all of the ops were on the same device, you wouldn't need to use read_value(), because TensorFlow doesn't make a copy of the variable when it is used as the input to an op on the same device. This can cause a lot of confusion—for example when you enqueue a variable to a queue!—and it's one of the reasons that we're working on enhancing the memory model for variables.)

这篇关于Tensorflow 0.12 版中 tf.Variable.ref() 的替代方案是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！