张量流中的参考边和非参考边有什么区别? [英] what's the difference between a ref edge and non-ref edge in tensorflow?

问题描述

正如标题所说，我想知道 TensorFlow 中参考边"和非参考边"之间的概念差异.

As the title says, I'm wondering about the conceptual difference between a "ref edge" and "non-ref edge" in TensorFlow.

我正在阅读 TensorFlow 中的图分区算法.这里(第826行graph_part of ) 是其中提到的注释非参考边":

I'm reading the graph partitioning algorithm in TensorFlow. Here (line 826 of graph_partition.cc) is the comment which mentions the "non-ref edge":

 825   // For a node dst, 'ref_recvs' remembers the recvs introduced by a ref
 826   // edge to dst. 'ref_control_inputs' remembers the inputs by a non-ref
 827   // edge to dst. We will add a control edge for every pair in
 828   // (ref_recvs x ref_control_inputs).
 829   std::vector<NodeDef*> ref_recvs;
 830   std::vector<string> ref_control_inputs;

有人可以更清楚地解释这种差异吗?非常感谢.

Can someone explain the difference more clearly? Thanks very much.

推荐答案

在 TensorFlow 中，大多数边是非引用"边，这意味着沿该边流动的值是一个常数.如果您将 TensorFlow 图中的顶点(操作)视为一个函数，您可以将非引用边视为表示函数参数 按值传递 在 C 或 C++ 等传统编程语言中.例如，操作 z = tf.matmul(x, y) 的输入和输出都是非参考边.

In TensorFlow, most edges are "non-ref" edges, which means that the value flowing along that edge is a constant. If you think of a vertex (operation) in a TensorFlow graph as a function, you can think of a non-ref edge as representing a function argument that is passed by value in a conventional programming language like C or C++. For example, the inputs to and outputs from the operation z = tf.matmul(x, y) are all non-ref edges.

TensorFlow 中的参考边"允许改变沿该边流动的值.继续函数的类比，ref 边代表一个函数参数，它是通过引用传递(我们从中取名为ref"边缘).引用边最常见的用途是在 tf.Variable 的当前内部实现中:内部Variablereferrer内核拥有一个可变缓冲区，并在 ref 边上输出对该缓冲区的引用.tf.assign(var, val) 等操作a> 期望他们的 var 参数沿着 ref 边传递，因为他们需要改变 var 中的值.

A "ref edge" in TensorFlow allows the value flowing along that edge to be mutated. Continuing the function analogy, a ref edge represents a function argument that is passed by reference (from which we take the name "ref" edge). The most common use of ref edges is in the current internal implementation of tf.Variable: the internal Variable kernel owns a mutable buffer, and outputs a reference to that buffer on a ref edge. Operations such as tf.assign(var, val) expect their var argument to be passed along ref edge, because they need to mutate the value in var.

图分区算法特别对待 ref 边，因为它们对应于可能随着图执行而改变的值.由于非参考边是一个常数值，TensorFlow 可以假设在两个设备之间交叉的同一操作中的所有非参考边都可以组合成单个边，从而节省网络/内存带宽.由于 ref 边上的值可能会发生变化(例如，如果在步骤中间更新了一个变量)，TensorFlow 必须小心不要合并这些边，以便远程设备可以看到新值.通过与 C/C++ 类比，TensorFlow 图分区器将 ref-edge 视为表示 volatile 变量，用于优化.

The graph partitioning algorithm treats ref edges specially because they correspond to values that could change as the graph executes. Since a non-ref edge is a constant value, TensorFlow can assume that all non-ref edges out of the same operation that cross between two devices can be combined into a single edge, which saves on network/memory bandwidth. Since the value on a ref edge can change (e.g. if a variable is updated in the middle of a step), TensorFlow must be careful not to combine the edges, so that the remote device can see the new value. By analogy with C/C++, the TensorFlow graph partitioner treats a ref-edge as representing a volatile variable, for the purposes of optimization.

最后，从上面的大量解释中可以看出，ref 边非常复杂，并且一直在努力将它们从 TensorFlow 执行模型中删除.替代的是资源类型的边"，它允许非张量值沿着边流动(统一变量、队列、阅读器和 TensorFlow 中的其他复杂对象)，以及将变量资源作为输入并读取其的显式操作值(作为非参考输出边沿).新的资源变量"的实现可以看这里是 Python 和 here/a>.

Finally, as you can tell from the amount of explanation above, ref edges are quite complicated, and there is an ongoing effort to remove them from the TensorFlow execution model. The replacement is "resource-typed edges", which allow non-tensor values to flow along an edge (unifying variables, queues, readers, and other complex objects in TensorFlow), and explicit operations that take a variable resource as input and read its value (as a non-ref output edge). The implementation of the new "resource variables" can be seen here in Python and here in C++.

这篇关于张量流中的参考边和非参考边有什么区别?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！