在CPU上运行Tensorflow时禁止OpenMP调试消息 [英] Suppress OpenMP debug messages when running Tensorflow on CPU

问题描述

在包含import tensorflow的Linux上运行Python程序(安装时未提供GPU支持)时，即使从未调用过tensorflow模块的函数，也会将一堆OpenMP调试消息写入stdout.这是摘录:

When running a Python program on Linux that includes import tensorflow (installed without GPU support), a bunch of OpenMP debug messages are written to stdout, even when no functions from the tensorflow module are ever called. Here's an excerpt:

OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids.
OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info
OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 0-3
OMP: Info #156: KMP_AFFINITY: 4 available OS procs
OMP: Info #157: KMP_AFFINITY: Uniform topology

在导入tensorflow之前设置os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'并不会消除这些消息，我该如何抑制它们(来自Python)?

Setting os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' before importing tensorflow does not get rid of these messages, how can I suppress them (from Python)?

推荐答案

底线:将KMP_WARNINGS envvar设置为错误值(0，FALSE，off或no).

Bottom line: Set KMP_WARNINGS envvar to a false value (0,FALSE, off or no).

(如果您使用多进程设置，请确保在生成子进程之前执行此操作，以使它们继承它.)

(If you use a multiple process setup, make sure to do this before you spawn child processes so that they inherit it.)

使用Google搜索"Affinity capable, using global cpuid leaf"(带引号-即完整短语)可找到 https://github.com/tensorflow/tensorflow 中搜索特别是对于短语或名为"openmp"的子模块引用什么都找不到)，但这是可以理解的，因为OpenMP是编译器实现的一部分(此文件明确声明是LLVM的一部分，即clang).该代码可能与您的程序使用的不是完全相同，而是它的另一个版本，但是我们可以假定常规逻辑是相同的，因此我们可以使用此代码进行导航.

Googling "Affinity capable, using global cpuid leaf" (with quotes -- i.e. as a full phrase) finds https://github.com/catboost/catboost/blob/master/contrib/libs/cxxsupp/openmp/i18n/en_US.txt . It's a part of an OpenMP implementation. This code is not a part of tensorflow (and searching within https://github.com/tensorflow/tensorflow specifically for the phrase or a submodule reference named "openmp" finds nothing) but this is understandable because OpenMP is a part of a compiler implementation (this file specifically claims to be a part of LLVM -- i.e. clang). This code may not be exactly what your program is using but another version of it, but we can assume that the general logic is the same, so we can use this code for navigation.

现在，搜索包含该短语的实体的引用(即AffUseGlobCpuidL11-这是消息的ID)，它说使消息if ( severity != kmp_ms_fatal && __kmp_generate_warnings == kmp_warnings_off )静音.通过浏览对__kmp_generate_warnings的引用来查找分配给它的位置，找到 https://github.com/catboost/catboost/blob/15712cfa704413d51618455326c30f5764956be5/contrib/libs/cxxsupp/openmp/kmp_settings.c#L944 并查找__kmp_stg_parse_warnings会找到.

Now, searching for references to the entity containing this phrase (namely, AffUseGlobCpuidL11 -- this is the ID of the message), then for a definition of KMP_INFORM, then for a definition of __kmp_msg ultimately finds the code doing the logging. It says that it only silences a message if ( severity != kmp_ms_fatal && __kmp_generate_warnings == kmp_warnings_off ). Looking through references to __kmp_generate_warnings to find where it's assigned finds https://github.com/catboost/catboost/blob/15712cfa704413d51618455326c30f5764956be5/contrib/libs/cxxsupp/openmp/kmp_settings.c#L944 and looking for __kmp_stg_parse_warnings finds https://github.com/catboost/catboost/blob/15712cfa704413d51618455326c30f5764956be5/contrib/libs/cxxsupp/openmp/kmp_settings.c#L4514 which suggests that name is "KMP_WARNINGS".

在这一点上，我认为应该记录该名称.因此，对其进行谷歌搜索将更快地告诉我用户应该如何设置"KMP_WARNINGS"以及允许的值是多少.我很失望-没有官方文件出炉. 另一个结果提示，尽管这是一个envvar，有效值为:使用" 0，" FALSE，".F.，" off，" no作为假值，" 1，" TRUE，".T."，"on"，"yes"作为真实值." 源代码还使用kmp_warnings_low作为可能的值，但名称__kmp_stg_parse_bool表示不可以，用户只能提供"true"和假"值，仅此而已.上面的过滤条件表明，除了kmp_warnings_off之外什么都不会产生影响，并且

At this point, I assumed that this name should be documented. So googling for it will tell me faster how that "KMP_WARNINGS" is supposed to be set by the user and what the allowed values for it are. I was disappointed -- no official documentation comes up. Another result suggests though that it's an envvar, and valid values are: "Use "0", "FALSE". ".F.", "off", "no" as false values, "1", "TRUE", ".T.", "on", "yes" as true values." The source code also uses kmp_warnings_low as a possible value but the name __kmp_stg_parse_bool suggests that no, the user can only provide a "true" and "false" value, nothing else. The above filter criterion says that nothing but kmp_warnings_off would have an effect anyway, and https://github.com/catboost/catboost/blob/15712cfa704413d51618455326c30f5764956be5/contrib/libs/cxxsupp/openmp/kmp_global.c#L116 says that low is the default if nothing is provided by the user.

这篇关于在CPU上运行Tensorflow时禁止OpenMP调试消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！