从源代码构建TensorFlow时，bazel规则在哪里生成`gen_io_ops.py`文件? [英] Where is the bazel rule generating the `gen_io_ops.py` file when building TensorFlow from sources?

问题描述

我试图确定从源代码构建TensorFlow时bazel如何生成 gen_io_ops 模块.

I'm trying to determine how the gen_io_ops module is generated by bazel when building TensorFlow from source.

在 tensorflow/python/ops/io_ops中. py ，有这段代码:

from tensorflow.python.ops.gen_io_ops
[...]

# used in the TextLineReader initialization
rr = gen_io_ops._text_line_reader_v2(...)

指的是bazel-genfiles/tensorflow/python/ops/gen_io_ops.py模块(由构建TensorFlow时的bazel生成).
_text_line_reader_v2是

referring to the bazel-genfiles/tensorflow/python/ops/gen_io_ops.py module (and generated by bazel when building TensorFlow).
The _text_line_reader_v2 is a wrapper of the TextLineReaderV2 defined in tensorflow/tensorflow/core/kernels/text_line_reader_op.cc.

据我了解，构建步骤如下:

As far as I understand, the build step are the followings:

1) text_line_reader_op的内核库内置于

其中tf_kernel_library基本上会查找 text_line_reader_op.c 文件并进行构建.

where tf_kernel_library basically looks for text_line_reader_op.c file and build it.

2)然后，:text_line_reader_op内核库用作io库的依赖项/master/tensorflow/core/kernels/BUILD"rel =" nofollow noreferrer>同一文件:

2) The :text_line_reader_op kernel library is then used as a dependency by the io library defined in the same file:

cc_library(
    name = "io",
    deps = [       
        ":text_line_reader_op", ...
    ],
)

我想io库现在包含TextLineReaderV2 kernel的定义.

I suppose the io library now contains the definition of the TextLineReaderV2kernel.

从我从答案中得到的结果来看，应该进行第三步 io库用于生成bazel-genfiles/tensorflow/python/ops/gen_io_ops.py模块中的python包装器.可以通过Basel中的tf_op_gen_wrapper_py规则或tf.load_op_library()方法来完成此文件的生成，但是似乎都没有涉及.

From what I get from this answer, there should be a third step where the io library is used to generate the python wrappers that are in the bazel-genfiles/tensorflow/python/ops/gen_io_ops.py module. This file generation can be done by the tf_op_gen_wrapper_py rule in Basel or by thetf.load_op_library() method, but none of them seem involved.

有人知道第三步在构建过程中的位置吗?

推荐答案

我终于明白了.

确实有对tf_op_gen_wrapper_py的调用，但隐藏在对tf_gen_op_wrapper_private_py的调用中:

There is indeed a call to tf_op_gen_wrapper_py but it's hidden in a call to tf_gen_op_wrapper_private_py:

def tf_gen_op_wrapper_private_py(name, out=None, deps=[],
                                 require_shape_functions=True,
                                 visibility=[]):
  if not name.endswith("_gen"):
    fail("name must end in _gen")
  [...]
  bare_op_name = name[:-4]
  tf_gen_op_wrapper_py(name=bare_op_name, ...

步骤如下.

在 tensorflow/tensorflow/python/BUILD 中，有这个规则

tf_gen_op_wrapper_private_py(
    name = "io_ops_gen",
    [...]
)

因此，在此规则中，_gen后缀将被删除(在tf_gen_op_wrapper_private_py中)，并且在tf_gen_op_wrapper_py中将添加gen_前缀，因此，该规则将生成gen_io_ops.py模块.

And so, in this rule the _gen suffix will be removed (in tf_gen_op_wrapper_private_py) and a gen_ prefix will be added in tf_gen_op_wrapper_py and therefore the gen_io_ops.py module will be generated by this rule.

这篇关于从源代码构建TensorFlow时，bazel规则在哪里生成`gen_io_ops.py`文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！