根据日期范围过滤日志文件条目 [英] Filter log file entries based on date range

问题描述

我的服务器具有异常高的CPU使用率，并且我可以看到Apache正在使用过多的内存. 我有一种感觉，我被一个IP所包围-也许您可以帮助我找到他?

My server is having unusually high CPU usage, and I can see Apache is using way too much memory. I have a feeling, I'm being DOS'd by a single IP - maybe you can help me find him?

我使用以下行来查找10个最活跃"的IP:

I've used the following line, to find the 10 most "active" IPs:

cat access.log | awk '{print $1}' |sort  |uniq -c |sort -n |tail

前5个IP对服务器的请求大约是普通"用户的200倍.但是，我无法确定这5位访问者是否只是非常频繁的访问者，还是他们在攻击服务器.

The top 5 IPs have about 200 times as many requests to the server, as the "average" user. However, I can't find out if these 5 are just very frequent visitors, or they are attacking the servers.

有没有办法将上述搜索指定到一个时间间隔，例如最近两个小时还是今天10点到12点?

Is there are way, to specify the above search to a time interval, eg. the last two hours OR between 10-12 today?

干杯！

2011年10月23日更新-我需要的命令:

在最近X个小时内(在两个小时内)获取条目

Get entries within last X hours [Here two hours]

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date) print Date FS $4}' access.log

在最近的X个小时内(在这里有两个小时)获得最活跃的IP地址

Get most active IPs within the last X hours [Here two hours]

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date) print $1}' access.log | sort  |uniq -c |sort -n | tail

获取相对时间范围内的条目

Get entries within relative timespan

awk -vDate=`date -d'now-4 hours' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print Date FS Date2 FS $4}' access.log

在绝对时间范围内获取条目

Get entries within absolute timespan

awk -vDate=`date -d '13:20' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'13:30' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print $0}' access.log 

在绝对时间内获得最活跃的IP

Get most active IPs within absolute timespan

awk -vDate=`date -d '13:20' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'13:30' +[%d/%b/%Y:%H:%M:%S` ' { if ($4 > Date && $4 < Date2) print $1}' access.log | sort  |uniq -c |sort -n | tail

推荐答案

是的，有多种方法可以做到这一点.这就是我要如何做的.对于初学者，无需通过管道传递cat的输出，只需使用awk打开日志文件.

yes, there are multiple ways to do this. Here is how I would go about this. For starters, no need to pipe the output of cat, just open the log file with awk.

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date {print Date, $0}' access_log

假设您的日志看起来像我的日志(它们是可配置的)，而不是日期存储在字段4中，并放在括号中.我在上面所做的就是在过去2小时内找到所有内容. Note the -d'now-2 hours'或现在翻译为减去两个小时的时间对我来说是这样的:[10/Oct/2011:08:55:23

assuming your log looks like mine (they're configurable) than the date is stored in field 4. and is bracketed. What I am doing above is finding everything within the last 2 hours. Note the -d'now-2 hours' or translated literally now minus 2 hours which for me looks something like this: [10/Oct/2011:08:55:23

所以我正在做的是存储两个小时前的格式化值，并与第四字段进行比较.条件表达式应该简单明了，然后打印日期，然后打印输出字段分隔符(OFS-在这种情况下为空格)，然后打印整行$ 0.您可以使用以前的表达式，仅打印$ 1(IP地址)

So what I am doing is storing the formatted value of two hours ago and comparing against field four. The conditional expression should be straight forward.I am then printing the Date, followed by the Output Field Separator (OFS -- or space in this case) followed by the whole line $0. You could use your previous expression and just print $1 (the ip addresses)

awk -vDate=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date {print $1}' | sort  |uniq -c |sort -n | tail

如果要使用范围，请指定两个日期变量并适当地构造表达式.

If you wanted to use a range specify two date variables and construct your expression appropriately.

因此，如果您希望在2-4小时之前找到一些东西，您的表情可能看起来像这样

so if you wanted do find something between 2-4hrs ago your expression might looks something like this

awk -vDate=`date -d'now-4 hours' +[%d/%b/%Y:%H:%M:%S` -vDate2=`date -d'now-2 hours' +[%d/%b/%Y:%H:%M:%S` '$4 > Date && $4 < Date2 {print Date, Date2, $4} access_log'

这是我回答的有关bash中日期的问题，您可能会觉得有帮助. 打印日期为当前一周(以bash表示)

Here is a question I answered regarding dates in bash you might find helpful. Print date for the monday of the current week (in bash)

这篇关于根据日期范围过滤日志文件条目的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！