检测日志文件轮换(在监视日志文件进行修改的同时) [英] Detect log file rotation (while watching log file for modification)

问题描述

我使用以下代码跟踪ssh登录:

I use the following code to track ssh log-ins:

def follow(thefile):
  thefile.seek(0,2)
  while True:
    line = thefile.readline()
    if not line:
      time.sleep(0.1)
      continue
    yield line

if __name__ == '__main__':
  logfile = open('/var/log/auth.log', 'r')
  loglines = follow(logfile)
  for line in loglines:
    print 'do something here'

我注意到该脚本在几天后突然停止工作.我没有收到任何错误，它没有终止，它只是停止工作，就像readline()永远不会返回.

I've noticed that this script suddenly stops working after a couple of days. I don't get any error, it doesn't terminate, it just stops working, as if readline() would never return.

所以我执行了echo 'test' >> auth.log.1，这确实被脚本处理了，因为前一段时间auth.log重命名为auth.log.1

So I executed a echo 'test' >> auth.log.1 and this indeed ends up getting processed by the script, because sometime ago auth.log got renamed to auth.log.1

如何跟踪这种日志轮换的时间并进行相应调整?

How can I track when such a log rotation happens and adjust accordingly?

推荐答案

使用e4c5的答案，我得到了此代码，该代码还解决了每秒多次调用readline()的问题.

Using e4c5's answer I ended up with this code, which also solves the issue of calling readline() multiple times per second.

在第一次调用期间，它会跳到文件末尾并等待修改.移动文件后，它将重新打开文件并读取全部内容，然后开始等待.

During the first invocation it skips to the end of the file and waits for modifications. When the file is moved, it reopens the file and reads the entire content, then starts to wait.

import os
import time
import traceback
import threading
import inotify.adapters

logfile = b'/var/log/auth.log'
#logfile = b'logfile.log'

##################################################################

def process(line, history=False):
  if history:
    print '=', line.strip('\n')
  else:
    print '>', line.strip('\n')

##################################################################

from_beginning = False
notifier = inotify.adapters.Inotify()
while True:
  try:
    #------------------------- check
    if not os.path.exists(logfile):
      print 'logfile does not exist'
      time.sleep(1)
      continue
    print 'opening and starting to watch', logfile
    #------------------------- open
    file = open(logfile, 'r')
    if from_beginning:
      for line in file.readlines():
        process(line, history=True)
    else:
      file.seek(0,2)
      from_beginning = True
    #------------------------- watch
    notifier.add_watch(logfile)
    try:
      for event in notifier.event_gen():
        if event is not None:
          (header, type_names, watch_path, filename) = event
          if set(type_names) & set(['IN_MOVE_SELF']): # moved
            print 'logfile moved'
            notifier.remove_watch(logfile)
            file.close()
            time.sleep(1)
            break
          elif set(type_names) & set(['IN_MODIFY']): # modified
            for line in file.readlines():
              process(line, history=False)
    except (KeyboardInterrupt, SystemExit):
      raise
    except:
      notifier.remove_watch(logfile)
      file.close()
      time.sleep(1)
    #-------------------------
  except (KeyboardInterrupt, SystemExit):
    break
  except inotify.calls.InotifyError:
    time.sleep(1)
  except IOError:
    time.sleep(1)
  except:
    traceback.print_exc()
    time.sleep(1)

##################################################################

这篇关于检测日志文件轮换(在监视日志文件进行修改的同时)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！