什么是使用python日志过滤不同记录器的正确方法？ [英] What is a correct way to filter different loggers using python logging?
问题描述
我的目的是通过 logging 提出的方式来进行分层过滤的多模块日志记录
作者Vinay Sajip，at至少据我猜测 - ）
您可以跳到我想要它如何工作
不幸的是，我很快就学会了使用日志记录工具的工作要比我的大多数其他语言经验复杂得多，而且我已经做了很多常见的（设计）错误，例如试图实现一个集中的单一Logger类记录多个模块，甚至像（使用Python记录器类为不同的日志级别生成多个日志）。但是显然有更好的设计空间，花时间去寻找和学习它可能会更糟糕。所以，现在我希望我走在正确的轨道上。否则Vinaj将不得不澄清其余的; - ）
我安排我的日志为：
- 每个python模块都有它的自己的记录器
- 每个记录器的名称都与其定义的模块名称相同，例如
logger = logging.getLogger（__ name __）像这样，每个模块中的代码可以使用自己的（本地定义的）记录器来发送将日志消息（logging.LogRecord）记录到处理程序（logging.Handler）中。 - 使用 logging.config ，以实现日志配置的完全灵活性（注意：在下面的代码中，我只是从 basicConfig ）
这种方法是推荐的方法，我同意它的可能的优点。例如，我可以使用完全限定的模块名称（代码中已经存在的命名层次结构）打开/关闭外部库的DEBUG。
现在有一个更高的级别控制我想使用logging.Filter类，以便能够过滤（允许）只有一个选定的子树内的记录器层次结构。
这一切都很好，但这里描述的过滤
过滤器实例用于执行LogRecords的任意过滤。
记录器和处理程序可以根据需要选择使用过滤器实例来过滤
记录。基本过滤器类只允许在记录器层次结构中某个点以下
的事件。例如，用AB初始化的过滤器
将允许由记录器AB，
ABC，ABCD，ABD等记录事件，但不允许记录A.BB， BAB等。如果用空字符串初始化
，所有事件都会被传递。
仍然不适合我。
＃不喜欢它
bar_logger.addFilter（logging.Filter（'foo'））
＃做一些工作。
foo（）
bar（）
只能打印
06-24 14:08 foo INFO hello from foo
我想如何工作
我想集中过滤它，即在我的根记录器不需要导入所有外部模块的记录器都在那里。
logging.root.addFilter（logging.Filter（'foo'））
打印
06-24 14:17 foo INFO hello from foo
06-24 14:17 bar信息hello from bar
一定有一些明显/愚蠢的错误，我想：我不想要任何来自 bar 记录器的消息。嘿，但是总结一下所有的人，找到它还有什么更好的办法？ ; - ）
我会试图找出bar_logger等待来自根记录器的决定的方式，然后发射任何东西。我只是希望这确实是应该如何工作的。
解决方案
添加过滤器而不是记录器：
handler.addFilter（logging.Filter（'foo'））
$ b $在您张贴的流程图中，请注意有两个钻石：
$ b $ b
- 连接到记录器的过滤器是否拒绝记录？
- 是否附加到 hander 拒绝记录如果将过滤器附加到根记录器，但通过例如foo或条形记录器启动LogRecord，则LogRecord不会被过滤，因为LogRecord会自由地通过foo或条形记录器，并且根记录器过滤器不会进入玩。 （再次查看流程图）
$ b 相比之下，由
basicConfig 定义的StreamHandler能够过滤任何LogRecord传递给它。 所以：将过滤器添加到处理程序而不是记录器中：
＃foo.py
导入日志
导入栏
$ b logger = logging.getLogger（'foo'）
def foo（）：
logger.info（'hello from foo'）
if __name__ =='__main__'：
＃日志记录设置。
logging.basicConfig（
level = logging.INFO，
format ='％（asctime）s％（name）-20s％（levelname）-8s％（message）s'，
datefmt ='％m-％d％H：％M'）
for logging.root.handlers：
handler.addFilter（logging.Filter（'foo'））
$ b $ f（）
bar.bar（）
p>
06-24 09:17 foo INFO hello from foo
如果您想允许记录名称以 foo 开头的记录程序或 bar ，但不能从任何其他记录器，您可以像这样创建一个白名单过滤器：
import logging
foo_logger = logging.getLogger（'foo'）
bar_logger = logging.getLogger（'bar'）
baz_logger = logging.getLogger（'baz'）
class whitelist（logging.Filter）：
def __init __（self，* whitelist）：
self.whitelist = [logging.Filter（na我）在白名单中的名称]
def filter（self，record）：
返回任何（f.filter（record）for f in self.whitelist）
logging.basicConfig（
level = logging.INFO，
format ='％（asctime）s％（name）-20s％（levelname）-8s％（message）s'，
date.php（白名单（'foo'，'bar'））$ b $日志文件（'％m-％d％H：％M'）
for logging.root.handlers：
handler.addFilter b
foo_logger.info（'hello from foo'）
＃06-24 09:41 foo INFO hello from foo
bar_logger.info（'hello from bar'）
＃ 06-24 09:41 bar信息hello from bar
baz_logger.info（'hello from baz'）
＃如果record.name不是以'foo'或'bar'开头，则自白名单过滤器以来没有输出
$ / code>
同样，您可以用这个黑名单记录名称：
类黑名单（白名单）：
def filter（self，record）：
不是白名单。 F ilter（self，record）
My purpose is to do a multi-module logging with hierarchical filtering
the way it is proposed by logging author Vinay Sajip, at least as far as I guess ;-)
You can skip to "How I want it to work"
Unfortunately, I learned very quickly that working with logging facilities is much more sophisticated than most of my other experience with the language and I already did a lot of common (design) mistakes, e.g. trying to achieve a centralized single Logger class logging for multiple modules or even schemes like (using Python logger class to generate multiple logs for different log levels). But apparently there is room for better design, and it might be worse spending time finding and learning it. So, right now I hope I am on the right track. Otherwise Vinaj will have to clarify the rest ;-)
I arrange my logging as this:
- Each python module has its own logger
- Each logger has a name same as the module where it is defined, e.g.
logger = logging.getLogger(__name__) - Like this, code inside each module can use its own (locally defined) logger to send logging messages (logging.LogRecord) to handlers (logging.Handler)
- Use logging.config to achieve full flexibility in configuring of the logging (Note: in the code below I just start with basicConfig)
Such approach is a recommended approach and I agree with its possible advantages. For example I can turn on/off DEBUG of external libraries using fully qualified module names (the naming hierarchy which already exists in the code).
Now to have a greater level of control I want to use logging.Filter class, to be able to filter (allow) only a selected subtree within the hierarchy of loggers.
This is all fine, but the filtering as described here
Filter instances are used to perform arbitrary filtering of LogRecords. Loggers and Handlers can optionally use Filter instances to filter records as desired. The base filter class only allows events which are below a certain point in the logger hierarchy. For example, a filter initialized with "A.B" will allow events logged by loggers "A.B", "A.B.C", "A.B.C.D", "A.B.D" etc. but not "A.BB", "B.A.B" etc. If initialized with the empty string, all events are passed.
is still not working for me.
My guess is that my lack of understanding the details behind LogRecords propagation is the source of the problem. Before jumping to code I want to show here a flow chart (from the cookbook tutorial which at first I somehow failed to immediately discover):
Example code
I start with two modules example, each uses it own named logger:
bar.py:
import logging
logger = logging.getLogger(__name__)
def bar():
logger.info('hello from ' + __name__)
foo.py:
import logging
from bar import bar, logger as bar_logger
logger = logging.getLogger('foo')
def foo():
logger.info('hello from foo')
if __name__ == '__main__':
# Trivial logging setup.
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(name)-20s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M'
)
# Do some work.
foo()
bar()
Logging is first constructed with logging.basicConfig (root logger, which was created after import logging by __main__ gets a stream handler attached to it, so that we have a console), is enabled (respective Logger.disabled=False) and both module loggers bar and foo propagate to the root logger (so we have three loggers in total).
print logger
print bar_logger
print logging.root
# Prints
#<logging.Logger object at 0x7f0cfd520790>
#<logging.Logger object at 0x7f0cfd55d710>
#<logging.RootLogger object at 0x7f0cfd520550>
The actual usecase is when bar is an external library that I want to silence (filter out).
How it works, but "me" don't like it
# Don't like it
bar_logger.addFilter(logging.Filter('foo'))
# Do some work.
foo()
bar()
prints only
06-24 14:08 foo INFO hello from foo
How I want it to work
I want to filter it out centrally, i.e. at my root logger w/o the need to import all the loggers of all the external modules out there.
logging.root.addFilter(logging.Filter('foo'))
prints
06-24 14:17 foo INFO hello from foo
06-24 14:17 bar INFO hello from bar
There must be some obvious/stupid mistake that I miss: I don't want any messages from bar logger. Hey, but what is a better way to find it than summarizing all on SO, folks? ;-)
I will try to find out the way for bar_logger to wait for decision from the root logger, before emitting anything. I just hope that this is indeed how it is supposed to work in the first place.
Solution
Add the filter to the handler rather than the logger:
handler.addFilter(logging.Filter('foo'))
Explanation
In the flow chart diagram you posted, notice there are two diamonds:
- Does a filter attached to logger reject the record?
- Does a filter attached to hander reject the record?
Thus, you get two swings at rejecting a LogRecord. If you attach the filter to the root logger, but initiate the LogRecord through, say, the foo or bar loggers, then the LogRecord does not get filtered because the LogRecord passes through the foo or bar loggers freely and the root logger filter never enters into play. (Look at the flow chart again.)
In contrast, the StreamHandler defined by basicConfig is capable of filtering any LogRecord passes to it.
So: add the filter to the handler rather than the logger:
# foo.py
import logging
import bar
logger = logging.getLogger('foo')
def foo():
logger.info('hello from foo')
if __name__ == '__main__':
# Trivial logging setup.
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(name)-20s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M')
for handler in logging.root.handlers:
handler.addFilter(logging.Filter('foo'))
foo()
bar.bar()
yields
06-24 09:17 foo INFO hello from foo
If you want to allow logging from loggers whose name begins with foo or bar, but not from any other loggers, you could create a whitelist filter like this:
import logging
foo_logger = logging.getLogger('foo')
bar_logger = logging.getLogger('bar')
baz_logger = logging.getLogger('baz')
class Whitelist(logging.Filter):
def __init__(self, *whitelist):
self.whitelist = [logging.Filter(name) for name in whitelist]
def filter(self, record):
return any(f.filter(record) for f in self.whitelist)
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s %(name)-20s %(levelname)-8s %(message)s',
datefmt='%m-%d %H:%M')
for handler in logging.root.handlers:
handler.addFilter(Whitelist('foo', 'bar'))
foo_logger.info('hello from foo')
# 06-24 09:41 foo INFO hello from foo
bar_logger.info('hello from bar')
# 06-24 09:41 bar INFO hello from bar
baz_logger.info('hello from baz')
# No output since Whitelist filters if record.name not begin with 'foo' or 'bar'
And similarly, you could blacklist logger names with this:
class Blacklist(Whitelist):
def filter(self, record):
return not Whitelist.filter(self, record)
这篇关于什么是使用python日志过滤不同记录器的正确方法？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
