为可变大小和固定大小的输入创建 TensorFlow 占位符有什么缺点吗? [英] Are there any downsides of creating TensorFlow placeholders for variable sized vs. fixed sized inputs?

问题描述

我想知道在为可变大小的输入(与固定大小的输入)创建 TensorFlow 占位符时是否有任何实质性的缺点(例如，关于计算效率、内存...)?

I am wondering is there are any substantial downsides (e.g., regarding computational efficiency, memory...) in creating TensorFlow placeholders for variable sized inputs (vs. fixed sized inputs)?

比如说，我正在做小批量学习并使用占位符初始化图形，我预先假设一个固定的 batch_size:

Say, I am doing mini-batch learning and initialize the graph with a placeholder where I assume a fixed batch_size upfront:

tf.placeholder(..., shape=[batch_size, ...])

或者，我可以初始化占位符变量，以便它接受可变大小的输入:

Alternatively, I can initialize the placeholder variable so that it accepts variable sized inputs:

tf.placeholder(..., shape=[None, ...])

我不太熟悉底层的 tensorflow 实现，但后者是否不必在每次迭代时检查维度、分配内存并创建新数组以解决我的 minibatch 大小在此期间发生变化的情况训练?那么，根据实现，如果我使用固定的批量维度，那会不会在计算上浪费?

I am not that familiar with the low level tensorflow implementations under the hood, but wouldn't the latter have to check dimensions, allocate memory, and create new arrays at each iteration to account for a case where my minibatch size changes during training? So, depending on the implementation, wouldn't that be computationally wasteful if I am working with a fixed batch dimension?

推荐答案

在提供完全定义的形状(可以产生更好的性能)和允许维度变化(这使得数据流图更易于重用)之间存在一定的紧张关系.正如您所怀疑的，使用可变形状的 tf.placeholder() 有一些缺点 在 TensorFlow 模型中表示输入的操作:

There is a definite tension between providing fully defined shapes (which can yield better performance) and allowing dimensions to vary (which makes the dataflow graph easier to reuse). As you suspect, there are some downsides to using variable-shaped tf.placeholder() ops to represent input in a TensorFlow model:

• 当形状完全已知时，TensorFlow 通常能够简化数据流图.例如，调用 tf.shape(x) 返回包含张量 x 的真实动态形状的 tf.Tensor.如果该形状在图构建时完全定义，TensorFlow 将用 tf.constant()，这将用于常量折叠优化以减少运行时完成的工作量.

• TensorFlow is often able to simplify the dataflow graph when shapes are fully known. For example, a call to tf.shape(x) returns a tf.Tensor containing the true dynamic shape of a tensor x. If that shape is fully defined at graph construction time, TensorFlow will replace the shape computation with a tf.constant(), and this will be used in constant folding optimizations to reduce the amount of work done at runtime.

作为一种极端情况，XLA 编译器要求在生成代码之前完全定义所有输入张量形状，以便它可以生成更高效的内核代码，其中数组边界(等)是编译时常量.XLA 为输入形状的每个组合重新编译内核代码，因此使用固定大小的张量将避免重新编译开销.

As an extreme case, the XLA compiler requires that all input tensor shapes be fully defined before code is generated, so that it can generate much more efficient kernel code where array bounds (etc.) are compile-time constants. XLA recompiles the kernel code for each combination of input shapes, so using fixed-size tensors will avoid the recompilation overhead.

TensorFlow 的内存分配器当前确实在每次调用 tf.Session.run() 时为每个中间和输出张量分配新数组.但是，如果底层内存分配器(GPU 内存的 BFC 分配器，以及 CPU 内存的 tcmalloc 或 jemalloc)具有静态分配请求，则它们的性能往往更好(因为可以从最近释放的缓冲区中满足请求).

TensorFlow's memory allocator currently does allocate new arrays for every intermediate and output tensor at each call to tf.Session.run(). However, the underlying memory allocators (the BFC allocator for GPU memory, and tcmalloc or jemalloc for CPU memory) tend to perform better if they have a static distribution of allocation requests (since the requests can be satisfied from buffers that were recently freed).

这篇关于为可变大小和固定大小的输入创建 TensorFlow 占位符有什么缺点吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！