在使用awk函数getline的bash指定的时间范围内提取日志文件数据 [英] extract data from log file in specified range of time with awk getline bash

问题描述

我正在寻找解析一个日志文件，发现什么，我需要在这个环节
<一href=\"http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time\">extract在指定的时间范围从日志文件数据

I was searching for parsing a log file and found what I need in this link extract data from log file in specified range of time

但是，最有用的答案（发表@Kent）：

But the most useful answer (posted by @Kent):

# this variable you could customize, important is convert to seconds. 
# e.g 5days=$((5*24*3600))
x=$((5*60))   #here we take 5 mins as example

# this line get the timestamp in seconds of last line of your logfile
last=$(tail -n1 logFile|awk -F'[][]' '{ gsub(/\//," ",$2); sub(/:/," ",$2); "date +%s -d \""$2"\""|getline d; print d;}' )

#this awk will give you lines you needs:
awk -F'[][]' -v last=$last -v x=$x '{ gsub(/\//," ",$2); sub(/:/," ",$2); "date +%s -d \""$2"\""|getline d; if (last-d<=x)print $0 }' logFile 

我认为错误在日期+％S -d .... 部分

是给了以下错误：

sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file
sh: -c: line 0: unexpected EOF while looking for matching `"'
sh: -c: line 1: syntax error: unexpected end of file

我花了很多时间试图解决之前，我问在这里，但没有找到任何解决方案。

I spend lot of time trying to solve before I ask here but didn't find any solution.

脚本将由 crontab中被称为来获得最后1分钟日志行和计数的IP是多少次在一分钟内上市，所以我可以，如果它检测到是攻击与否。这是另一项任务希望专家将帮助这里给需要code。在同样的问题。（我认为它可以在2线来解决）。

The script will be called by the crontab to get the last 1 min log lines and count how many times an ip is listed in one minute so I can detect if it is an attack or not. and this is another task hope that an expert will help giving the needed code here in the same question.(I think it can be solved in 2 lines).

推荐答案

从你输入立足这里，你可以使用脚本是这样的：

Basing from your input here, you could use a script like this:

#!/bin/bash

LOGFILE=/path/to/logfile

X=$(( 60 * 60 )) ## 1 Hour

function get_ts {
    DATE="${1%%\]*}"; DATE="${DATE##*\[}"; DATE=${DATE/:/ }; DATE=${DATE//\// }
    TS=$(date -d "$DATE" '+%s')
}

get_ts "$(tail -n 1 "$LOGFILE")"
LAST=$TS

while read -r LINE; do
    get_ts "$LINE"
    (( (LAST - TS) <= X )) && echo "$LINE"
done < "$LOGFILE"

将其保存到一个文件并更改LOGFILE值，然后用运行的bash script.sh 。

输出示例：

157.55.34.99 - - [06/Sep/2013:09:13:10 +0300] "GET /index.php HTTP/1.1" 200 16977 "-" "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)"
85.163.134.149 - - [06/Sep/2013:09:50:23 +0300] "GET /wap/wapicons/mnrwap.jpg HTTP/1.1" 200 1217 "http://mydomain.com/main.php" "Mozilla/5.0 (Linux; U; Android 4.1.2; en-gb; GT-I9082 Build/JZO54K) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30"

这篇关于在使用awk函数getline的bash指定的时间范围内提取日志文件数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！