根据日期范围过滤日志文件条目 [英] Filter log file entries based on date range

问题描述

我的服务器 CPU 使用率异常高，我可以看到 Apache 使用了太多内存.我有一种感觉，我正在被一个 IP DOS 攻击 - 也许你能帮我找到他?

My server is having unusually high CPU usage, and I can see Apache is using way too much memory. I have a feeling, I'm being DOS'd by a single IP - maybe you can help me find him?

我使用以下行来查找 10 个最活跃"的 IP:

I've used the following line, to find the 10 most "active" IPs:

cat access.log | awk '{print $1}' |sort  |uniq -c |sort -n |tail

前 5 个 IP 对服务器的请求大约是平均"用户的 200 倍.但是，我不知道这5个是访问频率很高的人，还是在攻击服务器.

The top 5 IPs have about 200 times as many requests to the server, as the "average" user. However, I can't find out if these 5 are just very frequent visitors, or they are attacking the servers.

有没有办法将上述搜索指定为一个时间间隔，例如.最近两个小时还是今天 10 点到 12 点之间?

Is there are way, to specify the above search to a time interval, eg. the last two hours OR between 10-12 today?

干杯！

2011 年 10 月 23 日更新 - 我需要的命令:

获取最近 X 小时内的条目 [这里是两个小时]

Get entries within last X hours [Here two hours]

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date) print Date FS $4}' access.log

获取最近 X 小时内最活跃的 IP [这里是两个小时]

Get most active IPs within the last X hours [Here two hours]

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date) print $1}' access.log | sort  |uniq -c |sort -n | tail

获取相对时间跨度内的条目

Get entries within relative timespan

awk -vDate=`date -d'now-4 hours' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print Date FS Date2 FS $4}' access.log

获取绝对时间跨度内的条目

Get entries within absolute timespan

awk -vDate=`date -d '13:20' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'13:30' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print $0}' access.log 

在绝对时间跨度内获取最活跃的 IP

Get most active IPs within absolute timespan

awk -vDate=`date -d '13:20' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'13:30' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print $1}' access.log | sort  |uniq -c |sort -n | tail

推荐答案

是的，有多种方法可以做到这一点.这是我将如何处理这个问题.对于初学者，不需要通过管道传输 cat 的输出，只需使用 awk 打开日志文件.

yes, there are multiple ways to do this. Here is how I would go about this. For starters, no need to pipe the output of cat, just open the log file with awk.

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date {print Date, $0}' access_log

假设您的日志看起来像我的(它们是可配置的)，而不是日期存储在字段 4 中.并且用括号括起来.我在上面所做的是在过去 2 小时内找到所有内容.注意 -d'now-2 hours' 或从字面上翻译现在减去 2 小时，对我来说看起来像这样:[10/Oct/2011:08:55:23>

assuming your log looks like mine (they're configurable) than the date is stored in field 4. and is bracketed. What I am doing above is finding everything within the last 2 hours. Note the -d'now-2 hours' or translated literally now minus 2 hours which for me looks something like this: [10/Oct/2011:08:55:23

所以我正在做的是存储两小时前的格式化值并与字段四进行比较.条件表达式应该是直截了当的.然后我打印日期，然后是输出字段分隔符(在这种情况下是 OFS——或空格)，然后是整行 $0.您可以使用之前的表达式并打印 $1(IP 地址)

So what I am doing is storing the formatted value of two hours ago and comparing against field four. The conditional expression should be straight forward.I am then printing the Date, followed by the Output Field Separator (OFS -- or space in this case) followed by the whole line $0. You could use your previous expression and just print $1 (the ip addresses)

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date {print $1}' | sort  |uniq -c |sort -n | tail

如果您想使用一个范围，请指定两个日期变量并适当地构建您的表达式.

If you wanted to use a range specify two date variables and construct your expression appropriately.

所以如果你想在 2-4 小时前找到一些东西，你的表情可能看起来像这样

so if you wanted do find something between 2-4hrs ago your expression might looks something like this

awk -vDate=`date -d'now-4 hours' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date && $4 < Date2 {print Date, Date2, $4} access_log'

这是我回答的有关 bash 日期的问题，您可能会觉得有帮助.打印星期一的日期当前周(在 bash 中)

Here is a question I answered regarding dates in bash you might find helpful. Print date for the monday of the current week (in bash)

这篇关于根据日期范围过滤日志文件条目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！