当它重新联机时，如何将未发送的日志发送到python日志记录服务器? [英] How to send unsent logs to python logging server when it comes back online?

问题描述

我已经在一台计算机上成功创建了中央python日志记录服务器，并且可以从多个RPis登录到该服务器.但是，当日志服务器关闭时，日志将丢失.有没有办法(以持久形式)存储日志，直到日志服务器再次可用?

I have successfully created a central python logging server on one computer and can log to it from multiple RPis. However when the logging server goes down the logs are lost. Is there a way to store the logs (in a persistent form) until the logging server becomes available again?

我看过一些程序，例如Graylog，Sentry和Logstash，但是看不到任何选择.

I have looked at a few programs such as graylog, sentry and logstash but couldn't see any option to do this.

任何帮助将不胜感激.

推荐答案

我想出了一个自定义套接字处理程序，该处理程序缓冲未发送的记录，直到传递为止.该缓冲区存储在二进制文件中，因此在程序重新启动或系统关闭期间会保留未发送的日志.

I've come up with a custom socket handler, which buffers unsent records until delivered. This buffer is being stored in a binary file, therefore the unsent logs are being kept during program restart or system shutdown.

import os, os.path
import logging.handlers
import pickle

class BufferingSocketHandler(logging.handlers.SocketHandler):
    def __init__(self, host, port, buffer_file):
        super().__init__(host, port)
        self._buffer = FileBuffer(buffer_file)

    @property  # getter only
    def buffer(self):
        return self._buffer

    def _emit(self, record):
        try:
            s = self.makePickle(record)
            self.send(s)
            return (self.sock is not None)
        except Exception:
            self.handleError(record)
            return False

    def emit(self, record):
        self.send_buffer()
        success = self._emit(record)
        if not success:
            self.buffer.append(record)

    def send_buffer(self):
        try:
            self.acquire()
            success = True
            for item in self.buffer:
                success &= self._emit(item)
            if success:
                self.buffer.flush()
        finally:
            self.release()


class FileBuffer:
    def __init__(self, fname):
        self.fname = fname

    @property
    def size(self):
        return int(os.path.isfile(self.fname) \
                and os.path.getsize(self.fname))

    def append(self, data):
        with open(self.fname, 'ba') as f:
            pickle.dump(data, f)

    def __iter__(self):
        if self.size > 0:
            try:
                with open(self.fname, 'br') as f:
                    while True:
                        yield pickle.load(f)
            except EOFError:
                return

    def flush(self):
        if self.size > 0:
            os.remove(self.fname)

在创建BufferingSocketHandler时，您将必须添加一个缓冲区文件名，在此文件名未保存的日志记录将一直保存到发送.数据正在使用 pickle 进行序列化，请参见有关示例，请参见上面的示例.但是，这是一个非常简单的实现，为了提高线程安全性，您可能需要添加一个数据库来存储日志记录.

When creating a BufferingSocketHandler you will have to add a buffer file name, where undelivered log records are being saved until delivery. Data is being serialized using pickle, see the FileBuffer class in the example above for details. However, this is a very bare implementation, for more thread safety you might want to add a database for storing log records.

该处理程序基于logging模块的 SocketHandler ，它的源可能很有趣以及了解这种修改背后的逻辑.

The handler is based on the logging module's SocketHandler, it's source could be interesting as well for understanding the logic behind this modification.

使用记录菜谱的示例，唯一需要做的更改就是导入此自定义处理程序并将其添加到根记录器中:

Using the logging Cookbook's example, the only changes necessary would be importing this custom handler and adding it to the root logger:

import logging, logging.handlers
from custom_handlers import BufferingSocketHandler

rootLogger = logging.getLogger('')
rootLogger.setLevel(logging.DEBUG)
socketHandler = BufferingSocketHandler('localhost',
                    logging.handlers.DEFAULT_TCP_LOGGING_PORT,
                    'logging-buffer.bin')
# don't bother with a formatter, since a socket handler sends
# the event as an unformatted pickle
rootLogger.addHandler(socketHandler)

此代码仅通过 Python 3 测试，我不知道它是否与Python-2兼容.

This code is tested with Python 3 only, I don't know if it is Python-2-compatible.

这篇关于当它重新联机时，如何将未发送的日志发送到python日志记录服务器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！