Logstash grok多行消息 [英] Logstash grok multiline message

问题描述

我的日志格式如下:

2014-06-19 02:26:05,556 INFO ok
2014-06-19 02:27:05,556 ERROR
 message:space exception
         at line 85
 solution:increase space
          remove files   

有两种类型的事件:

-与第一行一样登录

-如第二行一样登录多行

-log on multiple line like the second

我能够处理一个单行事件，但不能处理第二种事件，我想在其中将消息存储在一个变量中，将解决方案存储在另一个变量中.

I am able to process the one line event, but I am not able to process the second type, where I would like to stock the message in one variable and the solution in another.

这是我的配置:

input {
 file {
    path => ["logs/*"]
    start_position => "beginning"
    codec => multiline {
                   pattern => "^%{TIMESTAMP_ISO8601} "
                   negate => true
                   what => previous
    }       
 }
}
filter {
 #parsing of one line event
 grok {
 patterns_dir => "./patterns"
 match=>["message","%{TIMESTAMP_ISO8601:timestamp} %{WORD:level} ok"]
 }
#the parsing fail, so we assumed we are in multiline events, now I process them and I am stuck when I am getting to the new line.
if "_grokparsefailure" in [tags] {
 grok {
 patterns_dir => "./patterns"
 match=>["message","%{TIMESTAMP_ISO8601:timestamp} %{WORD:level}\r\n"]
 }
}

}

这就是我所做的，我想在控制台中输出以下内容:

So this is what I have done, and I would like to have in my console output the following:

{
"@timestamp" => "2014-06-19 00:00:00,000"
"path" => "logs/test.log"
"level"=>"INFO"
},
{
"@timestamp" => "2014-06-19 00:00:00,000"
"path" => "logs/test.log"
"level"=>"ERROR"
"message" => "space exception at line 85"
"solution"=>"increase space remove files"
}

具体来说，我想获得两个单词之间的所有表达式(消息变量为"message"和"solution"，解决方案变量为"solution"和event end)，无论该表达式是否在一行或多行上.

Concretely, I would like to get all the expression between two words ("message" and "solution" for the message variable, "solution" and the end of event for the solution variable), and that no matter if the expression is on one or multiple lines.

预先感谢

推荐答案

对于多行grok，最好对模式字符串使用特殊标志:

As for multiline grok, it's best to use special flag for pattern string:

grok {
    match => ["message", "(?m)%{SYSLOG5424LINE}"]
}

这篇关于Logstash grok多行消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！