Tensorflow:tf.nn.conv2d实际在哪里执行? [英] Tensorflow: Where is tf.nn.conv2d Actually Executed?

问题描述

我对tf.nn.conv2d(...)的Tensorflow实现感到好奇.要调用它，只需运行tf.nn.conv2d(...).但是，我要去尝试发现它的执行位置.代码如下(箭头指示最终调用的函数):

tf.nn.conv2d(...) -> tf.nn_ops.conv2d(...) -> tf.gen_nn_ops.conv2d(...) -> _op_def_lib.apply_op("Conv2D", ...) -> ?

我熟悉Tensorflow的LSTM实施，并且能够轻松地按照自己的意愿操纵它们.执行conv2d()计算的函数是用Python编写的吗?如果是，它在哪里?我可以看到跨步执行的位置和方式吗?

解决方案

TL; DR: 此处找到

.

.

您在问题中提到的功能链(从tf.nn.conv2d()向下)是用于构建 TensorFlow图的Python函数，但是这些函数不会调用实现.回想一下，在TensorFlow中，您首先构建符号图，然后执行. /p>

仅当您调用 传递其Tensor值，该值取决于某些卷积的结果.例如:

input = tf.placeholder(tf.float32)
filter = tf.Variable(tf.truncated_normal([5, 5, 3, 32], stddev=0.1)
conv = tf.nn.conv2d(input, filter, strides=[1, 1, 1, 1], padding='SAME')

result = sess.run(conv, feed_dict={input: ...})  # <== Execution happens here.

调用sess.run(...)告诉TensorFlow运行需要计算所有conv值的操作，包括卷积本身.从这里到实现的路径有些复杂，但是需要执行以下步骤:

1. sess.run()调用TensorFlow后端以获取conv的值.
2. 后端会修剪计算图以找出必须执行的节点，并将这些节点放置在适当的设备(CPU或GPU)上.
3. 使用执行器，指示每个设备执行其子图. >.
4. 执行程序最终通过调用其Compute()方法来调用与卷积运算符相对应的tensorflow::OpKernel.

"Conv2D" OpKernel是在此处实现的，其Compute()方法是此处.由于此操作对于许多工作负载而言对性能至关重要，因此实现起来相当复杂，但是基本思想是将计算分流到Eigen Tensor库(如果在CPU上运行)或cuDNN的优化GPU实现.

I am curious about the Tensorflow implementation of tf.nn.conv2d(...). To call it, one simply runs tf.nn.conv2d(...). However, I'm going down the rabbit hole trying to see where it is executed. The code is as follows (where the arrow indicates the function it ultimately calls):

tf.nn.conv2d(...) -> tf.nn_ops.conv2d(...) -> tf.gen_nn_ops.conv2d(...) -> _op_def_lib.apply_op("Conv2D", ...) -> ?

I am familiar with Tensorflow's implementation of LSTMs and the ability to easily manipulate them as one deems fit. Is the function that performs the conv2d() calculation written in Python, and if so, where is it? Can I see where and how the strides are executed?

解决方案

TL;DR: The implementation of tf.nn.conv2d() is written in C++, which invokes optimized code using either Eigen (on CPU) or the cuDNN library (on GPU). You can find the implementation here.

The chain of functions that you mentioned in the question (from tf.nn.conv2d() down) are Python functions for building a TensorFlow graph, but these do not invoke the implementation. Recall that, in TensorFlow, you first build a symbolic graph, then execute it.

The implementation of tf.nn.conv2d() is only executed happens when you call Session.run() passing a Tensor whose value depends on the result of some convolution. For example:

input = tf.placeholder(tf.float32)
filter = tf.Variable(tf.truncated_normal([5, 5, 3, 32], stddev=0.1)
conv = tf.nn.conv2d(input, filter, strides=[1, 1, 1, 1], padding='SAME')

result = sess.run(conv, feed_dict={input: ...})  # <== Execution happens here.

Invoking sess.run(...) tells TensorFlow to run all the ops that are neeeded to compute the value of conv, including the convolution itself. The path from here to the implementation is somewhat complicated, but goes through the following steps:

1. sess.run() calls the TensorFlow backend to fetch the value of conv.
2. The backend prunes the computation graph to work out what nodes must be executed, and places the nodes on the appropriate devices (CPU or GPU).
3. Each device is instructed to execute its subgraph, using an executor.
4. The executor eventually invokes the tensorflow::OpKernel that corresponds to the convolution operator, by calling its Compute() method.

The "Conv2D" OpKernel is implemented here, and its Compute() method is here. Because this op is performance critical for many workloads, the implementation is quite complicated, but the basic idea is that the computation is offloaded to either the Eigen Tensor library (if running on CPU), or cuDNN's optimized GPU implementation.

这篇关于Tensorflow:tf.nn.conv2d实际在哪里执行?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！