什么是“有状态对象"?在张量流中? [英] What is a "stateful object" in tensorflow?

问题描述

在文档的几个部分(例如 Dataset Iterators 此处a>) 有对 Stateful Objects 的引用.它们究竟是什么以及它们在图中扮演什么角色?

In several parts of the documentation (e.g. Dataset Iterators here) there are references to Stateful Objects. What exactly are they and what role do they play in the graph?

澄清一下，在数据集文档中有一个 one_shot_iterator 的例子，因为它是无状态的:

To clarify, in the Dataset documentation there's an example with the one_shot_iterator that works because it's stateless:

dataset = tf.data.Dataset.range(100)
iterator = dataset.make_one_shot_iterator()

是什么让迭代器无状态?

what makes the iterator stateless?

推荐答案

正如其他人提到的，有状态的对象是那些持有状态的对象.现在，在 TensorFlow 术语中，状态是在对 tf.Session.run.最常见和最基本的有状态对象是变量.您可以调用一次 run 来更新模型的参数，这些参数是变量，它们将在下一次调用 run 时保持其分配的值.这与大多数操作不同；例如，如果您有一个加法运算，它需要两个张量并输出第三个张量，则不会保存它在一次调用 run 中计算的输出值.实际上，即使您的图形仅包含具有常量值的操作，每次调用 run 时都会评估张量操作，即使结果始终相同.然而，当您赋值给一个变量时，它会棒"(顺便说一句，获取相应的内存，如果您选择这样做，则在检查点上进行序列化).

As others have mentioned, stateful objects are those holding a state. Now, a state, in TensorFlow terms, is some value or data that is saved between different calls to tf.Session.run. The most common and basic kind of stateful objects are variables. You can call run once to update a model's parameters, which are variables, and they will maintain their assigned value for the next call to run. This is different to most operations; for example, if you have an addition operation, which takes two tensors and outputs a third one, the output value that it computes in one call to run is not saved. Indeed, even if your graph consists only on operations with constant values, tensor operations will be evaluated every time you call run, even though the result will always be the same. When you assign a value to a variable, however, it will "stick" (and, by the way, take the corresponding memory and, if you choose so, be serialized on checkpoints).

数据集迭代器也是有状态的.当你在一次运行中获得一条数据时，它就会被消耗掉，然后在下一次运行中你得到另一条数据；迭代器记住"它在两次运行之间的位置.这就是为什么，类似于初始化变量的方式，您可以初始化迭代器(当它们支持时)，将它们重置回已知状态.

Dataset iterators are also stateful. When you get a piece of data in one run then it is consumed, and then in the next run you get a different piece of data; the iterator "remembers" where it was between runs. That is why, similarly to how you initialize variables, you can initialize iterators (when they support it), to reset them back to a known state.

从技术上讲，另一种有状态的对象是随机操作.人们通常认为随机操作是随机的，但实际上它们拥有一个随机数生成器，该生成器确实具有在运行之间保持的状态，如果您提供种子，那么当您开始时它们将处于明确定义的状态会议.但是，据我所知，没有任何方法可以在同一会话中将随机操作重置为其初始状态.

Technically speaking, another kind of stateful objects is random operations. One usually regards random operations as, well, random, but in reality they hold a random number generator that does have a state which is held between runs, and if you provide a seed then they will be in a well-defined state when you start the session. However there is not, as far as I know, any way to reset random operations to their initial state within the same session.

请注意，有状态"一词经常(当人们不是特别指代 TensorFlow 时)在稍微不同的意义上使用，或者在不同的抽象级别上使用.例如，循环神经网络 (RNN) 通常被认为是有状态的，因为从概念上讲，它们有一个内部状态，随着他们收到的每个输入而改变.但是，当您在 TensorFlow 中创建 RNN 时，该内部状态不一定必须在有状态对象中！与任何其他类型的神经网络一样，TensorFlow 中的 RNN 原则上会有一些参数或权重，通常存储在可训练变量中 - 因此，在 TensorFlow 术语中，所有可训练模型，无论是否为 RNN，都具有训练参数的状态对象.但是，RNN 的内部状态在 TensorFlow 中表示为每次运行时获得的输入状态值和输出状态值(请参阅 tf.nn.dynamic_rnn)，您可以在每次运行时从零"状态开始，而忘记最终输出状态.当然，如果您愿意，您也可以将输入状态作为变量的值，然后将输出状态写回到该变量中，然后您的 RNN 内部状态对于 TensorFlow 将是有状态的"；也就是说，您将能够在一次运行中处理一些数据，并在下一次运行中从您离开的地方拿起"(这可能有意义，也可能没有意义，具体取决于具体情况).我知道这可能有点令人困惑，但我希望它是有道理的.

Note that frequently (when one is not referring to TensorFlow in particular) the term "stateful" is used in a slightly different sense, or at a different level of abstraction. For example, recurrent neural networks (RNNs) are generally said to be stateful, because, conceptually, they have an internal state that changes with every input they receive. When you make a RNN in TensorFlow, however, that internal state does not necessarily have to be in a stateful object! Like any other kind of neural network, RNNs in TensorFlow will in principle have some parameters or weights, typically stored in trainable variables - so, in TensorFlow terms, all trainable models, RNN or not, have stateful objects for the trained parameters. However, the internal state of the RNN is represented in TensorFlow with an input state value and and output state value that you get on each run (see tf.nn.dynamic_rnn), and you can just start with a "zero" state on each run and forget about the final output state. Of course, you can also, if you want, take the input state to be the value of a variable, and the write the output state back to that variable, and then your RNN internal state will be "stateful" for TensorFlow; that is, you would be able to process some data in one run and the "pick up where you left it" in the next run (which may or may not make sense depending on the case). I understand this can be a bit confusing but I hope it makes sense.

这篇关于什么是“有状态对象"?在张量流中?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！