获取两个日期之间的日志文件条目，只有致命的日期 [英] Get log file entries between two dates, only fatal ones

问题描述

我在网上搜索，所以没有人愿意做这项工作. 给定一个简单的Apache日志文件，例如

I searched the web, SO and nothing wants to do the job. Given a simple apache logfile like

[Fri Jun 22 11:46:13 2018] [error] [client xxxxxxxx] PHP Parse error:  syntax error, unexpected end of file in yyyyyyyyyyyyyyyyy on line 3554, referer: wwwwwwwwwwwwwwwwwwwwwwww
[Fri Jun 22 14:09:37 2018] [error] [client xxxxxxxx] PHP Fatal error:  Call to undefined function date_mysql2german() in yyyyyyyyyyyyyyyyy on line 156, referer: wwwwwwwwwwwwwwwwwwwwwwww
[Mon Jun 25 17:03:37 2018] [error] [client xxxxxxxx] PHP Warning:  mysql_num_rows() expects parameter 1 to be resource, null given in yyyyyyyyyyyyyyyyy on line 1409, referer: wwwwwwwwwwwwwwwwwwwwwwww
[Tue Jun 26 11:46:26 2018] [error] [client xxxxxxxx] PHP Warning:  mysql_num_rows() expects parameter 1 to be resource, boolean given in yyyyyyyyyyyyyyyyy on line 9390, referer: wwwwwwwwwwwwwwwwwwwwwwww
[Tue Jun 26 11:46:26 2018] [error] [client xxxxxxxx] PHP Warning:  mysql_num_rows() expects parameter 1 to be resource, boolean given in yyyyyyyyyyyyyyyyy on line 9432, referer: wwwwwwwwwwwwwwwwwwwwwwww

我需要提取最近5分钟的所有"PHP警告:"行(完整行).

I need to extract all "PHP Warning:" lines (full line) of the last 5 minutes.

这是我到目前为止尝试过的

Here is what I tried so far

awk 
-v Date="$(date "+[%a %b %d %H:%M:%S %Y")" \ 
-v Date2="$(date --date="5 minutes ago" "+[%a %b %d %H:%M:%S %Y")" \
'$4 > Date && $4 < Date2' /var/log/apache2/apache2/my_log_file.log

但是，即使这样(例如，没有grep表示警告")也不会返回任何内容.粘贴的日志条目及其日期/时间仅是示例-实际上，我在最近5分钟内确实有警告，因此它应明确返回一些内容.

But even that (without grep for "Warning" e.g.) does not return anything. The pasted log entries and their date/times are just examples - in reality I do have Warnings within the last 5 minutes, so it should definitly return something.

关于如何更改这些内容并使它正常工作的任何想法? 提前致谢 亚历克斯

Any ideas on how to change this stuff and get it to work? Thanks in advance Alex

推荐答案

问题是您比较字符串的方式过于文字化.日期按数据排序，但awk按字典顺序进行比较(按日期顺序"Jan"<"2月"，按字符串顺序则不是).这里可以采用多种方法，但是我建议在UNIX时代进行比较.

The problem is that you compare your strings way too literal. The dates are ordered by data, but awk compares lexicographical ("Jan" < "Feb" datewise but string wise it is not). There are various approaches that can be taken here, but I suggest to make the comparisons in UNIX epoch time.

$ tend=$(date "+%s")
$ tstart=$(date --date="5 minutes ago" "+%s")
$ awk -F '[][]' '!/PHP Warning/{next}
                 { cmd="date --date=\""$2"\" \"+%s\""
                   time=((cmd | getline line) > 0 ? line : -1)
                   close(cmd) }
                 (time == -1) { exit 1 }
                 (tend <= time && time <= tstart)
                ' tstart=$tstart tend=$tend <logfile>

注意:如果文件很大，这将对date进行大量调用.

note: this will execute a large number of calls to date if your file is big.

另一种方法可能是调用 mktime 从GNU awk或将您的时间字符串重新格式化为 yyyymmddHHMMSS .这使您可以对字符串使用字典顺序:

A different approach might be calling mktime from GNU awk or reformatting your time string as yyyymmddHHMMSS. This allows you to use lexicographical ordering for strings :

$ tstart=$(date -d="5 minutes ago" "+%Y%m%d%H%M%S"")
$ tend=$(date "+%Y%m%d%H%M%S"")
$ awk 'BEGIN{ month["Jan"]="01"; month["Feb"]="02"; month["Mar"]="03"
            month["Arp"]="04"; month["May"]="05"; month["Jun"]="06"
            month["Jul"]="07"; month["Aug"]="08"; month["Sep"]="09"
            month["Oct"]="10"; month["Nov"]="11"; month["Dec"]="12" }
      !/PHP Warning/{next}
      { time=$4; gsub(/:/,"",time); year=substr($5,1,4);
        date=sprintf(%4s%2s%0.2d%6s,year,month[$2],$3,time) }
      }
      (tstart <= date && tend <= date)
     ' tend=$tend tstart=$tstart  <logfile>

或根据 Ed的建议莫顿:

$ awk '!/PHP Warning/{next}
       { year=substr($5,1,4)
         month=(index("JanFebMarAprMayJunJulAugSepOctNovDec",$2)+2)/3
         time=$4; gsub(/:/,"",time);
         date=sprintf(%4s%0.2d%0.2d%6s,year,month,$3,time)
       }
       (tstart <= date && tend <= date)
      ' tend=$end tstart=$tstart <logfile>

可以在此处找到相关文章:用于匹配日志文件自定义日期格式的正则表达式

A related post can be found here: Regex to match logfile custom date formats

这篇关于获取两个日期之间的日志文件条目，只有致命的日期的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！