在一个上下文中而不是另一个上下文中禁用python模块的日志记录 [英] Disable logging for a python module in one context but not another

问题描述

我有一些使用 requests 模块与日志API进行通信的代码.但是，请求本身通过 urllib3 进行记录.自然，我需要禁用日志记录，以便对日志记录API的请求不会导致日志无限循环.因此，在该模块中，我将进行日志记录调用，并执行 logging.getLogger("requests").setLevel(logging.CRITICAL)来使常规请求日志静音.

I have some code that uses the requests module to communicate with a logging API. However, requests itself, through urllib3, does logging. Naturally, I need to disable logging so that requests to the logging API don't cause an infinite loop of logs. So, in the module I do the logging calls in, I do logging.getLogger("requests").setLevel(logging.CRITICAL) to mute routine request logs.

但是，此代码旨在加载和运行任意用户代码.由于python logging 模块显然使用全局状态来管理给定记录器的设置，因此我担心用户的代码可能会重新登录并引起问题，例如，如果他们天真的在其请求中使用了requests模块代码而没有意识到我出于某种原因禁用了日志记录.

However, this code is intended to load and run arbitrary user code. Since the python logging module apparently uses global state to manage settings for a given logger, I am worried the user's code might turn logging back on and cause problems, for instance if they naively use the requests module in their code without realizing I have disabled logging for it for a reason.

当从我的代码上下文执行请求模块时，如何禁用请求模块的日志记录，但从用户的角度来看，不影响该模块的记录器状态?某种类型的上下文管理器可以使对管理器中的代码的日志记录的调用静音.能够使用唯一的 __ name __ 加载请求模块，以便记录器使用其他名称也可以工作，尽管有点麻烦.不过，我找不到一种可以做这两种事情的方法.

How can I disable logging for the requests module when it is executed from the context of my code, but not affect the state of the logger for the module from the perspective of the user? Some sort of context manager that silences calls to logging for code within the manager would be ideal. Being able to load the requests module with a unique __name__ so the logger uses a different name could also work, though it's a bit convoluted. I can't find a way to do either of these things, though.

遗憾的是，该解决方案将需要处理多个线程，因此在程序上关闭日志记录，然后运行API调用，然后再将其重新打开将不会起作用，因为全局状态已发生变化.

Regrettably, the solution will need to handle multiple threads, so procedurally turning off logging, then running the API call, then turning it back on will not work as global state is mutated.

推荐答案

我认为我为您提供了一个解决方案:

I think I've got a solution for you:

logging 模块是内置的线程安全的:

日志记录模块旨在为线程安全的，没有任何特殊要求客户需要完成的工作.它通过使用来实现螺纹锁；有一个锁可以序列化对模块的访问共享数据，并且每个处理程序还创建一个锁以序列化访问到其基础I/O.

The logging module is intended to be thread-safe without any special work needing to be done by its clients. It achieves this though using threading locks; there is one lock to serialize access to the module’s shared data, and each handler also creates a lock to serialize access to its underlying I/O.

幸运的是，它公开了通过公共API提及的第二个锁: Handler.acquire() 可让您获取特定日志处理程序的锁定(和

Fortunately, it exposes the second lock mentioned though a public API: Handler.acquire() lets you acquire a lock for a particular log handler (and Handler.release() releases it again). Acquiring that lock will block all other threads that try to log a record that would be handled by this handler until the lock is released.

这允许您以线程安全的方式操纵处理程序的状态.需要注意的是:由于它是作为处理程序的I/O操作的锁，因此只能在 emit()中获取该锁.因此，只有一条记录通过过滤器和日志级别将其记录下来，并由特定处理程序发出后，该锁才会被获取.这就是为什么我必须对处理程序进行子类化并创建 SilencableHandler .

This allows you to manipulate the handler's state in a thread-safe way. The caveat is this: Because it's intended as a lock around the I/O operations of the handler, the lock will only be acquired in emit(). So only once a record makes it through filters and log levels and would be emitted by a particular handler will the lock be acquired. That's why I had to subclass a handler and create the SilencableHandler.

因此，想法是这样的:

• 为 requests 模块获取最顶层的记录器，并停止其传播
• 创建您的自定义 SilencableHandler 并将其添加到请求记录器中
• 使用 Silenced 上下文管理器有选择地使 SilencableHandler
静音

• Get the topmost logger for the requests module and stop propagation for it
• Create your custom SilencableHandler and add it to the requests logger
• Use the Silenced context manager to selectively silence the SilencableHandler

main.py

main.py

from Queue import Queue
from threading import Thread
from usercode import fetch_url
import logging
import requests
import time


logging.basicConfig(level=logging.INFO)
log = logging.getLogger(__name__)


class SilencableHandler(logging.StreamHandler):

    def __init__(self, *args, **kwargs):
        self.silenced = False
        return super(SilencableHandler, self).__init__(*args, **kwargs)

    def emit(self, record):
        if not self.silenced:
            super(SilencableHandler, self).emit(record)


requests_logger = logging.getLogger('requests')
requests_logger.propagate = False
requests_handler = SilencableHandler()
requests_logger.addHandler(requests_handler)


class Silenced(object):

    def __init__(self, handler):
        self.handler = handler

    def __enter__(self):
        log.info("Silencing requests logger...")
        self.handler.acquire()
        self.handler.silenced = True
        return self

    def __exit__(self, exc_type, exc_value, traceback):
        self.handler.silenced = False
        self.handler.release()
        log.info("Requests logger unsilenced.")


NUM_THREADS = 2
queue = Queue()

URLS = [
    'http://www.stackoverflow.com',
    'http://www.stackexchange.com',
    'http://www.serverfault.com',
    'http://www.superuser.com',
    'http://travel.stackexchange.com',
]


for i in range(NUM_THREADS):
    worker = Thread(target=fetch_url, args=(i, queue,))
    worker.setDaemon(True)
    worker.start()

for url in URLS:
    queue.put(url)


log.info('Starting long API request...')

with Silenced(requests_handler):
    time.sleep(5)
    requests.get('http://www.example.org/api')
    time.sleep(5)
    log.info('Done with long API request.')

queue.join()

usercode.py

usercode.py

import logging
import requests
import time


logging.basicConfig(level=logging.INFO)
log = logging.getLogger(__name__)


def fetch_url(i, q):
    while True:
        url = q.get()
        response = requests.get(url)
        logging.info("{}: {}".format(response.status_code, url))
        time.sleep(i + 2)
        q.task_done()

示例输出:

(请注意，不会记录对 http://www.example.org/api 的调用，并且在最初的10秒钟内将阻止所有尝试记录请求的线程).

(Notice how the call to http://www.example.org/api isn't logged, and all threads that try to log requests are blocked for the first 10 seconds).

INFO:__main__:Starting long API request...
INFO:__main__:Silencing requests logger...
INFO:__main__:Requests logger unsilenced.
INFO:__main__:Done with long API request.
Starting new HTTP connection (1): www.stackoverflow.com
Starting new HTTP connection (1): www.stackexchange.com
Starting new HTTP connection (1): stackexchange.com
Starting new HTTP connection (1): stackoverflow.com
INFO:root:200: http://www.stackexchange.com
INFO:root:200: http://www.stackoverflow.com
Starting new HTTP connection (1): www.serverfault.com
Starting new HTTP connection (1): serverfault.com
INFO:root:200: http://www.serverfault.com
Starting new HTTP connection (1): www.superuser.com
Starting new HTTP connection (1): superuser.com
INFO:root:200: http://www.superuser.com
Starting new HTTP connection (1): travel.stackexchange.com
INFO:root:200: http://travel.stackexchange.com

线程代码基于Doug Hellmann在线程和

Threading code is based on Doug Hellmann's articles on threading and queues.

这篇关于在一个上下文中而不是另一个上下文中禁用python模块的日志记录的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！