检测日志文件轮换(同时查看日志文件进行修改) [英] Detect log file rotation (while watching log file for modification)

问题描述

我使用以下代码来跟踪 ssh 登录:

def follow(thefile):thefile.seek(0,2)而真:line = thefile.readline()如果不是行:时间.睡眠(0.1)继续屈服线如果 __name__ == '__main__':logfile = open('/var/log/auth.log', 'r')日志行 = 跟随(日志文件)对于日志行中的行:打印在这里做点什么"

我注意到这个脚本在几天后突然停止工作.我没有收到任何错误，它不会终止，它只是停止工作，就好像 readline() 永远不会返回一样.

所以我执行了一个 echo 'test' >>auth.log.1 并且这确实最终被脚本处理，因为前一段时间 auth.log 被重命名为 auth.log.1

如何跟踪此类日志轮换发生的时间并进行相应调整?

解决方案

使用 e4c5 的答案我最终得到了这段代码，这也解决了每秒多次调用 readline() 的问题.

在第一次调用期间，它跳到文件末尾并等待修改.当文件被移动时，它重新打开文件并读取整个内容，然后开始等待.

导入操作系统导入时间导入回溯进口螺纹导入 inotify.adapters日志文件 = b'/var/log/auth.log'#logfile = b'logfile.log'###############################################################定义过程(行，历史=假):如果历史:打印 '=', line.strip('
')别的:打印 '>', line.strip('
')###############################################################from_beginning = 假通知程序 = inotify.adapters.Inotify()而真:尝试:# -  -  -  -  -  -  -  -  -  -  -  - - 查看如果不是 os.path.exists(logfile):打印日志文件不存在"时间.sleep(1)继续打印打开并开始观看"，日志文件# -  -  -  -  -  -  -  -  -  -  -  - - 打开文件 = 打开(日志文件，'r')如果从_开始:对于 file.readlines() 中的行:过程(行，历史=真)别的:file.seek(0,2)from_beginning = 真# -  -  -  -  -  -  -  -  -  -  -  - - 手表notifier.add_watch(日志文件)尝试:对于 notifier.event_gen() 中的事件:如果事件不是无:(header, type_names, watch_path, filename) = 事件如果设置(type_names)&set(['IN_MOVE_SELF']): # 移动打印日志文件已移动"notifier.remove_watch(日志文件)文件.close()时间.sleep(1)休息elif 集(type_names)&set(['IN_MODIFY']): # 修改对于 file.readlines() 中的行:过程(行，历史=假)除了(键盘中断，系统退出):增加除了:notifier.remove_watch(日志文件)文件.close()时间.sleep(1)#-------------------------除了(键盘中断，系统退出):休息除了inotify.calls.InotifyError:时间.sleep(1)除了 IOError:时间.sleep(1)除了:traceback.print_exc()时间.sleep(1)###############################################################

I use the following code to track ssh log-ins:

def follow(thefile):
  thefile.seek(0,2)
  while True:
    line = thefile.readline()
    if not line:
      time.sleep(0.1)
      continue
    yield line

if __name__ == '__main__':
  logfile = open('/var/log/auth.log', 'r')
  loglines = follow(logfile)
  for line in loglines:
    print 'do something here'

I've noticed that this script suddenly stops working after a couple of days. I don't get any error, it doesn't terminate, it just stops working, as if readline() would never return.

So I executed a echo 'test' >> auth.log.1 and this indeed ends up getting processed by the script, because sometime ago auth.log got renamed to auth.log.1

How can I track when such a log rotation happens and adjust accordingly?

解决方案

Using e4c5's answer I ended up with this code, which also solves the issue of calling readline() multiple times per second.

During the first invocation it skips to the end of the file and waits for modifications. When the file is moved, it reopens the file and reads the entire content, then starts to wait.

import os
import time
import traceback
import threading
import inotify.adapters

logfile = b'/var/log/auth.log'
#logfile = b'logfile.log'

##################################################################

def process(line, history=False):
  if history:
    print '=', line.strip('
')
  else:
    print '>', line.strip('
')

##################################################################

from_beginning = False
notifier = inotify.adapters.Inotify()
while True:
  try:
    #------------------------- check
    if not os.path.exists(logfile):
      print 'logfile does not exist'
      time.sleep(1)
      continue
    print 'opening and starting to watch', logfile
    #------------------------- open
    file = open(logfile, 'r')
    if from_beginning:
      for line in file.readlines():
        process(line, history=True)
    else:
      file.seek(0,2)
      from_beginning = True
    #------------------------- watch
    notifier.add_watch(logfile)
    try:
      for event in notifier.event_gen():
        if event is not None:
          (header, type_names, watch_path, filename) = event
          if set(type_names) & set(['IN_MOVE_SELF']): # moved
            print 'logfile moved'
            notifier.remove_watch(logfile)
            file.close()
            time.sleep(1)
            break
          elif set(type_names) & set(['IN_MODIFY']): # modified
            for line in file.readlines():
              process(line, history=False)
    except (KeyboardInterrupt, SystemExit):
      raise
    except:
      notifier.remove_watch(logfile)
      file.close()
      time.sleep(1)
    #-------------------------
  except (KeyboardInterrupt, SystemExit):
    break
  except inotify.calls.InotifyError:
    time.sleep(1)
  except IOError:
    time.sleep(1)
  except:
    traceback.print_exc()
    time.sleep(1)

##################################################################

这篇关于检测日志文件轮换(同时查看日志文件进行修改)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！