解析Apache2的错误日志与神交的Logstash [英] Parse Apache2 Error logs with Grok for Logstash
问题描述
我试着来分析我的Apache2错误日志和有一点麻烦IM ..它不似乎是相匹配的过滤器。林pretty确保时间戳件是错的，但林不知道，我真的不能找到任何文件来弄明白。此外，有没有办法得到的是在 fields.errmsg 我 @message ？
登录
[周三6月26日22点13分22秒2013] [错误] [客户端10.10.10.100] PHP致命错误：未捕获的异常'\\富\\酒吧'
托运人配置
输入{
文件{
'路径'= GT; /var/log/apache2/*-error.log
'型'=＆GT; Apache的错误
}}过滤{
神交{
键入= GT; Apache的错误
模式=＆GT; \\ [％{HTTPDATE：时间戳} \\] \\ [％{WORD：类} \\] \\ [％{WORD：发端}％{IP：clientip} \\]％{GREEDYDATA：ERRMSG}
}
}
输出{
标准输出{}
Redis的{
DATA_TYPE'=＆GT; 名单
'主机'= GT; logstash.server.net
钥匙= GT; logstash
}
}
嗨！
我知道我有点迟到了，但在这里它去！
我创建的系统上的 /etc/logstash/patterns.d / 目录，thew一个名为 Apache的错误它包含：
APACHE_ERROR_TIME％{} DAY％{MONTH}％{} MONTHDAY％{TIME}％{} YEAR
APACHE_ERROR_LOG \\ [％{APACHE_ERROR_TIME：时间戳} \\] \\ [％{LOGLEVEL：日志等级} \\]（？：\\ [客户端％{IPORHOST：clientip} \\]）{0,1}％{GREEDYDATA：ERRORMSG}
/etc/logstash/patterns.d / 将在 logstash 配置引用如下：
神交{
patterns_dir =＆GT; [/etc/logstash/patterns.d]
比赛=＆GT; [消息，％{} APACHE_ERROR_LOG]
}
您可以在神交调试测试它，像亚当在他的评论中提及。似乎工作只是你送的小样日志条目罚款。公司一直致力于pretty扎实我与我的客户之一。
上面的图案把最后的消息 ERRORMSG 字段。所以我只是删除的消息字段。
这是我目前用我logstash配置：
过滤器{
如果[类型] ==apache_error_log{ 神交{
patterns_dir =＆GT; [/etc/logstash/patterns.d]
比赛=＆GT; [消息，％{} APACHE_ERROR_LOG]
} 如果！（_ grokparsefailure，在[标签]）{ 变异{
remove_field =＆GT; [ 信息 ]
add_field =＆GT; [timestamp_submitted，％{@时间戳}]
} 日期{
＃尝试拉从时间戳字段时间戳（以上解析
＃神交）。 Apache的时间格式如下：18月/ 8月/ 2011：05：44：34 -0700
＃周六2月08日6时31分09秒2014年
比赛=＆GT; [时间戳，EEE MMM DD HH：MM：SS YYYY]
remove_field =＆GT; [时间戳]
} geoip的{
来源=＆GT; clientip
}
}
}
}
请注意，我用的类型的 apache_error_log 而不是 Apache的错误。
给它的射门。我很想知道，如果这对你的作品和其他人！
干杯！
Im trying to parse my apache2 error log and im having a bit of trouble.. It doesnt seem to be matching the filter. Im pretty sure the timestamp piece is wrong, but im not sure, and i cant really find any documentation to figure it out. Also, is there a way to get what is in fields.errmsg to me @message?
Log
[Wed Jun 26 22:13:22 2013] [error] [client 10.10.10.100] PHP Fatal error: Uncaught exception '\Foo\Bar'
Shipper Config
input {
file {
'path' => '/var/log/apache2/*-error.log'
'type' => 'apache-error'
}
}
filter {
grok {
type => "apache-error"
pattern => "\[%{HTTPDATE:timestamp}\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}"
}
}
output {
stdout {}
redis {
'data_type' => 'list'
'host' => 'logstash.server.net'
'key' => 'logstash'
}
}
Ahoy!
I know I'm a little late to the party, but here it goes!
I created a /etc/logstash/patterns.d/ directory on the system and thew a file named apache-error in it containing:
APACHE_ERROR_TIME %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}
APACHE_ERROR_LOG \[%{APACHE_ERROR_TIME:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:errormsg}
/etc/logstash/patterns.d/ will be referenced in the logstash configuration as follows:
grok {
patterns_dir => [ "/etc/logstash/patterns.d" ]
match => [ "message", "%{APACHE_ERROR_LOG}" ]
}
You can test it out at Grok Debugger, like Adam mentioned in his comment. Seems to work just fine with the sample log entry you send. Has been working pretty solidly me with one of my clients.
The pattern above puts the final message in errormsg field. So I just delete the message field.
This is what I am currently using in my logstash configuration:
filter {
if [type] == "apache_error_log" {
grok {
patterns_dir => [ "/etc/logstash/patterns.d" ]
match => [ "message", "%{APACHE_ERROR_LOG}" ]
}
if !("_grokparsefailure" in [tags]) {
mutate {
remove_field => [ "message" ]
add_field => ["timestamp_submitted", "%{@timestamp}"]
}
date {
# Try to pull the timestamp from the 'timestamp' field (parsed above with
# grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700"
# Sat Feb 08 06:31:09 2014
match => [ "timestamp", "EEE MMM dd HH:mm:ss yyyy" ]
remove_field => [ "timestamp" ]
}
geoip {
source => "clientip"
}
}
}
}
Note that I am using a type of apache_error_log instead of apache-error.
Give it s shot. I would love to know if this works for you and others!
Cheers!
这篇关于解析Apache2的错误日志与神交的Logstash的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
