TensorFlow tf.group 忽略依赖项? [英] TensorFlow tf.group ignoring dependencies?

问题描述

继更早的问题之后，似乎tf.group 确实忽略了依赖关系.这是一个简单的独立示例(我已在 Python 2.7 和 TensorFlow 1.1 上运行它):

Following on from an earlier question, it seems tf.group is indeed ignoring dependencies. Here's a simple stand-alone example (I have run it on Python 2.7 with TensorFlow 1.1):

import tensorflow as tf
from tensorflow.python.ops import control_flow_ops

xs = [tf.constant(x) for x in range(10)]
xs = [tf.Print(x, [x]) for x in xs]
dependency = None
dxs = []

for x in xs:
    if dependency is None:
        dependency = x
    else:
        dependency = control_flow_ops.with_dependencies([dependency], x)

    dxs.append(dependency)

print_all_op = tf.group(*dxs)

with tf.Session() as session:
    session.run(print_all_op)

预期输出:

2017-05-29 15:11:53.961221: I tensorflow/core/kernels/logging_ops.cc:79] [0]
2017-05-29 15:11:53.961236: I tensorflow/core/kernels/logging_ops.cc:79] [1]
2017-05-29 15:11:53.961255: I tensorflow/core/kernels/logging_ops.cc:79] [2]
2017-05-29 15:11:53.961237: I tensorflow/core/kernels/logging_ops.cc:79] [3]
2017-05-29 15:11:53.961262: I tensorflow/core/kernels/logging_ops.cc:79] [4]
2017-05-29 15:11:53.961263: I tensorflow/core/kernels/logging_ops.cc:79] [5]
2017-05-29 15:11:53.961268: I tensorflow/core/kernels/logging_ops.cc:79] [6]
2017-05-29 15:11:53.961272: I tensorflow/core/kernels/logging_ops.cc:79] [7]
2017-05-29 15:11:53.961274: I tensorflow/core/kernels/logging_ops.cc:79] [8]
2017-05-29 15:11:53.961221: I tensorflow/core/kernels/logging_ops.cc:79] [9]

实际输出(每次运行代码都不同):

Actual output (different each time the code is run):

2017-05-29 15:16:26.279655: I tensorflow/core/kernels/logging_ops.cc:79] [0]
2017-05-29 15:16:26.279655: I tensorflow/core/kernels/logging_ops.cc:79] [9]
2017-05-29 15:16:26.279697: I tensorflow/core/kernels/logging_ops.cc:79] [3]
2017-05-29 15:16:26.279660: I tensorflow/core/kernels/logging_ops.cc:79] [1]
2017-05-29 15:16:26.279711: I tensorflow/core/kernels/logging_ops.cc:79] [8]
2017-05-29 15:16:26.279713: I tensorflow/core/kernels/logging_ops.cc:79] [4]
2017-05-29 15:16:26.279723: I tensorflow/core/kernels/logging_ops.cc:79] [5]
2017-05-29 15:16:26.279663: I tensorflow/core/kernels/logging_ops.cc:79] [2]
2017-05-29 15:16:26.279724: I tensorflow/core/kernels/logging_ops.cc:79] [7]
2017-05-29 15:16:26.279728: I tensorflow/core/kernels/logging_ops.cc:79] [6]

tf.group 说明为何忽略依赖项的文档.

There's nothing in the tf.group documentation to indicate why dependencies are ignored.

有没有考虑依赖关系的 tf.group 替代方案?

Is there an alternative to tf.group that does consider dependencies?

切换到使用 tf.control_dependencies 而不是 tensorflow.python.ops.control_flow_ops.with_dependencies 没有帮助:

Switching to use tf.control_dependencies instead of tensorflow.python.ops.control_flow_ops.with_dependencies doesn't help:

import tensorflow as tf

xs = [tf.constant(x) for x in range(10)]
xs = [tf.Print(x, [x]) for x in xs]
dependency = None
dxs = []

for x in xs:
    if dependency is None:
        dependency = x
    else:
        with tf.control_dependencies([dependency]):
            dependency = x

    dxs.append(dependency)

print_all_op = tf.group(*dxs)

with tf.Session() as session:
    session.run(print_all_op)

推荐答案

我认为问题在于初始代码创建了由 control_flow_ops.with_dependencies 隐式创建的虚拟身份操作之间的依赖关系，而不是实际的<代码>tf.Print 操作.Tensorflow 似乎只确保依赖列表中的 ops 已经被执行，而其他前面的 ops 的顺序是不固定的.在上面的示例中，依赖项是在 control_flow_ops.with_dependencies 创建的虚拟身份操作上创建的:

I think the problem is that the initial code creates dependencies between dummy identity ops implicitly created by control_flow_ops.with_dependencies and not the actual tf.Print ops. Tensorflow seems to only ensure that the ops in the dependency list have been already executed but the order of other preceding ops is not fixed. In the above example, the dependencies are created on the dummy identity ops created by control_flow_ops.with_dependencies:

    dependency = control_flow_ops.with_dependencies([dependency], x)

应该等价于:

        with tf.control_dependencies([dependency]):
            dependency = tf.identity(x)

因此，这里的依赖关系是在 tf.identity 操作而不是 tf.Print 操作之间创建的.tf.Print 操作可以以任何顺序运行，严格的排序仅适用于 tf.identity 操作.我认为使用 control_flow_ops.with_dependencies 无法实现所需的行为.相反，必须使用 with tf.control_dependencies 代替(正如操作已经建议的那样):

Thus, the dependencies here are created between the tf.identity ops and not the tf.Print ops. The tf.Print ops can be run in any order, the strict ordering is only on the tf.identity ops. I don't think it is possible to achieve the desired behavior with control_flow_ops.with_dependencies. Instead one has to use with tf.control_dependencies instead (as already suggested by the op):

xs = [tf.constant(x) for x in range(10)]
dependency = None
dxs = []

for x in xs:
    if dependency is None:
        dependency = tf.Print(x, [x])
    else:
        with tf.control_dependencies([dependency]):
            dependency = tf.Print(x, [x])

    dxs.append(dependency)

这篇关于TensorFlow tf.group 忽略依赖项?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！