如何抑制冗长的Tensorflow日志记录? [英] How to suppress verbose Tensorflow logging?

问题描述

我正在用鼻子测试对Tensorflow代码进行单元测试，但它会产生如此大量的详细输出，从而使其无用.

I'm unittesting my Tensorflow code with nosetests but it produces such amount of verbose output that makes it useless.

以下测试

import unittest
import tensorflow as tf

class MyTest(unittest.TestCase):

    def test_creation(self):
        self.assertEquals(True, False)

使用nosetests运行时会创建大量无用的日志记录:

when run with nosetests creates a huge amount of useless logging:

FAIL: test_creation (tests.test_tf.MyTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/cebrian/GIT/thesis-nilm/code/deepmodels/tests/test_tf.py", line 10, in test_creation
    self.assertEquals(True, False)
AssertionError: True != False
-------------------- >> begin captured logging << --------------------
tensorflow: Level 1: Registering Const (<function _ConstantShape at 0x7f4379131c80>) in shape functions.
tensorflow: Level 1: Registering Assert (<function no_outputs at 0x7f43791319b0>) in shape functions.
tensorflow: Level 1: Registering Print (<function _PrintGrad at 0x7f4378effd70>) in gradient.
tensorflow: Level 1: Registering Print (<function unchanged_shape at 0x7f4379131320>) in shape functions.
tensorflow: Level 1: Registering HistogramAccumulatorSummary (None) in gradient.
tensorflow: Level 1: Registering HistogramSummary (None) in gradient.
tensorflow: Level 1: Registering ImageSummary (None) in gradient.
tensorflow: Level 1: Registering AudioSummary (None) in gradient.
tensorflow: Level 1: Registering MergeSummary (None) in gradient.
tensorflow: Level 1: Registering ScalarSummary (None) in gradient.
tensorflow: Level 1: Registering ScalarSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering MergeSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering AudioSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering ImageSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering HistogramSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering HistogramAccumulatorSummary (<function _ScalarShape at 0x7f4378f042a8>) in shape functions.
tensorflow: Level 1: Registering Pack (<function _PackShape at 0x7f4378f047d0>) in shape functions.
tensorflow: Level 1: Registering Unpack (<function _UnpackShape at 0x7f4378f048c0>) in shape functions.
tensorflow: Level 1: Registering Concat (<function _ConcatShape at 0x7f4378f04938>) in shape functions.
tensorflow: Level 1: Registering ConcatOffset (<function _ConcatOffsetShape at 0x7f4378f049b0>) in shape functions.

......

从ipython控制台使用tensorflow似乎并不那么冗长:

whereas using tensorflow from the ipython console doesn't seem that verbose:

$ ipython
Python 2.7.11+ (default, Apr 17 2016, 14:00:29) 
Type "copyright", "credits" or "license" for more information.

IPython 4.2.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import tensorflow as tf
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcuda.so locally
I tensorflow/stream_executor/dso_loader.cc:108] successfully opened CUDA library libcurand.so locally

In [2]:

运行鼻子测试时如何抑制以前的记录?

How could I suppress the former logging when running nosetests?

推荐答案

2.0更新(10/8/19) 设置TF_CPP_MIN_LOG_LEVEL应该仍然可以工作(请参见v0.12 +更新中的以下内容)，但是当前存在一个未解决的问题(请参见

2.0 Update (10/8/19) Setting TF_CPP_MIN_LOG_LEVEL should still work (see below in v0.12+ update), but there is currently an issue open (see issue #31870). If setting TF_CPP_MIN_LOG_LEVEL does not work for you (again, see below), try doing the following to set the log level:

import tensorflow as tf
tf.get_logger().setLevel('INFO')

此外，请参阅 tf.autograph.set_verbosity 上的文档设置签名日志消息的详细程度-例如:

In addition, please see the documentation on tf.autograph.set_verbosity which sets the verbosity of autograph log messages - for example:

# Can also be set using the AUTOGRAPH_VERBOSITY environment variable
tf.autograph.set_verbosity(1)

v0.12 +更新(5/20/17)，通过TF 2.0 +:

在TensorFlow 0.12+中，根据此问题，您现在可以通过环境变量称为TF_CPP_MIN_LOG_LEVEL；它默认为0(显示所有日志)，但可以在Level列下设置为以下值之一.

In TensorFlow 0.12+, per this issue, you can now control logging via the environmental variable called TF_CPP_MIN_LOG_LEVEL; it defaults to 0 (all logs shown) but can be set to one of the following values under the Level column.

  Level | Level for Humans | Level Description                  
 -------|------------------|------------------------------------ 
  0     | DEBUG            | [Default] Print all messages       
  1     | INFO             | Filter out INFO messages           
  2     | WARNING          | Filter out INFO & WARNING messages 
  3     | ERROR            | Filter out all messages      

请参见以下使用Python的通用操作系统示例:

See the following generic OS example using Python:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  # or any {'0', '1', '2'}
import tensorflow as tf

更详尽地说，您还调用了Python的级别 tf_logging 模块，例如摘要操作，张量板，各种估计器等.

To be thorough, you call also set the level for the Python tf_logging module, which is used in e.g. summary ops, tensorboard, various estimators, etc.

# append to lines above
tf.logging.set_verbosity(tf.logging.ERROR)  # or any {DEBUG, INFO, WARN, ERROR, FATAL}

对于1.14，如果您不更改以使用v1 API，您将收到警告:

For 1.14 you will receive warnings if you do not change to use the v1 API as follows:

# append to lines above
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)  # or any {DEBUG, INFO, WARN, ERROR, FATAL}

---

对于TensorFlow或TF-Learn日志的早期版本(v0.11.x或更低版本):

查看以下页面以获取有关TensorFlow日志记录的信息；在新的更新中，您可以将日志记录的详细程度设置为DEBUG，INFO，WARN，ERROR或FATAL.例如:

View the page below for information on TensorFlow logging; with the new update, you're able to set the logging verbosity to either DEBUG, INFO, WARN, ERROR, or FATAL. For example:

tf.logging.set_verbosity(tf.logging.ERROR)

该页面还翻过了可与TF-Learn模型一起使用的监视器. 这是页面.

The page additionally goes over monitors which can be used with TF-Learn models. Here is the page.

尽管

不会阻止所有日志记录(仅TF-Learn).我有两种解决方案；一种是技术上正确的"解决方案(Linux)，另一种是重建TensorFlow.

This doesn't block all logging, though (only TF-Learn). I have two solutions; one is a 'technically correct' solution (Linux) and the other involves rebuilding TensorFlow.

script -c 'python [FILENAME].py' | grep -v 'I tensorflow/'

有关其他信息，请参见

For the other, please see this answer which involves modifying source and rebuilding TensorFlow.

这篇关于如何抑制冗长的Tensorflow日志记录?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！