如何将消息从不同的线程记录到不同的文件? [英] How to log messages from different threads to different files?

问题描述

我有一个 Driver.py 脚本，它根据给定的输入调用多个 threads .线程基本上是运行所选对象的模块.因此， Driver.py 可能会调用 thread_1.run()， thread_2.run()， thread_3.run()，并继续其过程.

I have a Driver.py scripts where it calls multiple threads based on the given inputs. Threads are basically runs a module of a selected object. So Driver.py may call thread_1.run(), thread_2.run(), thread_3.run(), and continue its process.

Driver.py 将其输出记录到main.log文件夹中，在此我希望线程将其输出记录到每个文件的唯一文件名中. Driver.py 和 Threads 也使用在不同文件中定义的通用模块，它们还记录信息.

Driver.py logs its output into a main.log folder, where I want threads to log their output into unique filenames for each. Driver.py and Threads also uses common modules that are defined on differnt files, where they also logs information.

我首先在 Driver 上调用 setup_load("main.log")，然后在每个 Thread 中调用，称为 setup_load(f以及"thread_ {jobid} .log").我意识到当调用 Thread 时，现在Driver.py会写入线程的日志文件中.我可能在Thread中使用了一个不同的记录器，但是当线程调用另一个模块时，由于这些通用模块正在使用 import logging ，它们会写入根记录器的定义文件名.

I call setup_load("main.log") first on Driver, afterwards in each Thread, called setup_load(f"thread_{jobid}.log") as well. I realize that when Thread is called, now Driver.py writes into thread's log file. I may use a differnt logger inside Thread but when the Thread calls another modules, since those common modules are using import logging they write into the root logger's defined filename.

=> 是否可以将消息从不同的线程记录到不同的文件?我在SO上找到了多个答案(例如 )，但其中没有一个涵盖了在另一个文件上调用另一个模块时的情况，他们将如何找到可以使用的 logger .

=> Is it possible to log messages from different threads to different files? I found multiple answers on SO (for example), but non of them covers when another module is called on a different file, how would they can find out that which logger they can use.

=> 所以我面临的问题是，当我在一个线程中更改 logging.basicConfig 的文件路径时，每个线程都使用相同的基础记录器，它会影响所有线程和驱动程序中的类，因为它们都在使用它.

=> So the problem I am facing is since every thread is using the same underlying logger, when I change the file path of the logging.basicConfig in one thread, it affects the class across all threads and the driver, since they're all using it.

=> 如何从线程或驱动程序中调用不同模块的功能来了解应该选择哪个记录器?

=> How would be functions from different modules called from the thread or driver would understand which logger should it choose?

关于有关如何使用不同类和导入即时记录Python更改文件句柄的部分进行了讨论并建议使用解决方案.

Comment section on How to change filehandle with Python logging on the fly with different classes and imports has a discussion and recommended solution.

@Martijn Pieters:

@Martijn Pieters:

下一个选项:创建每个线程处理程序，为每个处理程序提供一个过滤器在logrecord的 thread 属性上进行过滤.将过滤器附加到任何其他设置了 thread 设置

next option: create per-thread handlers, give each of them a filter that filters on the logrecord thread attribute. Attach a filter to any other handlers that returns False for logrecords with thread set

推荐答案

是的，您可以将日志条目从不同的线程定向到不同的文件.您需要:

Yes, you can direct log entries from different threads to different files. You'll need to:

• 创建一个日志过滤器，该过滤器可以通过以下方式过滤记录他们的 LogRecord.thread 或 LogRecord.threadName 属性
• 创建一个过滤器，该过滤器不接受具有特定或所有线程ID的记录.
• 为每个线程创建一个日志处理程序，为其提供一个日志过滤器，该过滤器仅接受特定线程的日志记录.
• 将忽略线程日志记录的过滤器附加到任何其他处理程序.

• Create a log filter that can filter records by their LogRecord.thread or LogRecord.threadName attribute
• Create a filter that does not accept records with specific or all thread ids.
• Create a log handler per thread, giving it a log filter that only accepts logrecords for their specific thread.
• Attach the filter that ignores log records for your threads to any other handlers.

过滤时，您可以选择过滤线程ID(由 threading.get_ident() )或线程名称(无论您以 name 作为参数传递给

When filtering, you have the choice between filtering on thread id (the value returned by threading.get_ident()) or thread name (whatever you passed in as the name argument to the Thread() object). If you have a pattern for your thread names, this is where you'd use it.

创建自定义过滤器非常简单:

Creating a custom filter is easy enough:

import threading
from logging import Filter

class ThreadFilter(Filter):
    """Only accept log records from a specific thread or thread name"""

    def __init__(self, threadid=None, threadname=None):
        if threadid is None and threadname is None:
            raise ValueError("Must set at a threadid and/or threadname to filter on")
        self._threadid = threadid
        self._threadname = threadname

    def filter(self, record):
        if self._threadid is not None and record.thread != self._threadid:
            return False
        if self._threadname is not None and record.threadName != self._threadname:
            return False
        return True

class IgnoreThreadsFilter(Filter):
    """Only accepts log records that originated from the main thread"""

    def __init__(self):
        self._main_thread_id = threading.main_thread().ident

    def filter(self, record):
        return record.thread == self._main_thread_id

如果要匹配特定的模式，请相应地调整代码.

If you want to match a specific pattern, adjust the code accordingly.

这篇关于如何将消息从不同的线程记录到不同的文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！