如何处理不匹配的Logstash grok过滤器 [英] How to handle non-matching Logstash grok filters

问题描述

我想知道什么是我的Logstash Grok过滤器最好的方法。我有一些针对特定日志条目的过滤器，并不适用于所有条目。不适用的总是生成_grokparsefailure标签。例如，我有一个grok过滤器是每个日志条目，它工作正常。然后我有另一个过滤器是用于回溯错误消息。回溯过滤器为每个没有回溯的日志条目抛出一个grokparsefailure。

如果不存在，我宁愿让它通过规则一个匹配而不是添加parsefailure标签。我使用parsefailure标签来查找不正确解析的东西，而不是简单地不匹配特定过滤器的东西。也许这只是命名解析失败，让我。对我来说，这意味着有什么错误的过滤器（例如格式不正确），而不是它不匹配。

所以问题是，我该如何处理？ （ab）使用tag_on_failure选项，将其设置为nothing []

使用if traceback in message / li>

别的东西我没有考虑过？

提前。

编辑

  if [message] =〜/ took\s\d + / {
 grok {
 patterns_dir => / etc / logstash / patterns
 match => [message，took \s +（？< servicetime> [\d\。] +）] 
 add_tag => [stats，servicetime] 
} 
} 
  

反馈虽然。什么被认为是最佳做法在这里？

解决方案

如果可能的话，我会去与条件包装，就像你正在使用的。如果您的应用程序只生成几种不同的行格式，您可以使用多个匹配模式与 //logstash.net/docs/1.3.2/filters/grokrel =noreferrer> grok过滤器。默认情况下，过滤器将处理第一次成功的匹配：

  grok {
 patterns_dir => ./patterns
 match => [
message，％{BASE_PATTERN}％{EXTRA_PATTERN}，
message，％{BASE_PATTERN}，
message，％{SOME_OTHER_PATTERN}
 
 
 
 
 
 
 $ b如果你的逻辑不那么简单（也许你需要检查相同的情况不止一次）， grep过滤器可以用来添加一个标签。就像这样：
 
 
  grep {
 drop => false #grep通常会丢弃不匹配的事件
 match => [message，/ took'\\s\d + /] 
 add_tag => has_traceback
} 
 
 
 ... 
 
如果[tags]中有has_traceback{
 ... 
} 
  
 
I am wondering what the best approach to take with my Logstash Grok filters.  I have some filters that are for specific log entries, and won't apply to all entries.  The ones that don't apply always generate _grokparsefailure tags.  For example, I have one grok filter that's for every log entry and it works fine.  Then I have another filter that's for error messages with tracebacks.  The traceback filter throws a grokparsefailure for every single log entry that doesn't have a traceback.

I'd prefer to have it just pass the rule if there isn't a match instead of adding the parsefailure tag. I use the parsefailure tag to find things that aren't parsing properly, not things that simply didn't match a particular filter.  Maybe it's just the nomenclature "parse failure" that gets me.  To me that means there's something wrong with the filter (e.g. badly formatted), not that it didn't match.

So the question is, how should I handle this?


Make the filter pattern optional using ?
(ab)use the tag_on_failure option by setting it to nothing []
make the filter conditional using something like "if traceback in message"
something else I'm not considering?


Thanks in advance.

EDIT

I took the path of adding a conditional around the filter:
    if [message] =~ /took\s\d+/ {
        grok {
            patterns_dir => "/etc/logstash/patterns"
            match => ["message", "took\s+(?<servicetime>[\d\.]+)"]
            add_tag => [ "stats", "servicetime" ]
        }
    }
Still interested in feedback though.  What is considered "best practice" here?
解决方案
When possible, I'd go with a conditional wrapper just like the one you're using. Feel free to post that as an answer!

If your application produces only a few different line formats, you can use multiple match patterns with the grok filter. By default, the filter will process up to the first successful match:
grok {
    patterns_dir => "./patterns"
    match => [
        "message", "%{BASE_PATTERN} %{EXTRA_PATTERN}",
        "message", "%{BASE_PATTERN}",
        "message", "%{SOME_OTHER_PATTERN}"
    ]
}
If your logic is less straightforward (maybe you need to check the same condition more than once), the grep filter can be useful to add a tag. Something like this:
grep {
    drop => false #grep normally drops non-matching events
    match => ["message", "/took\s\d+/"]
    add_tag => "has_traceback"
}


...

if "has_traceback" in [tags] {
    ...
}


                        
这篇关于如何处理不匹配的Logstash grok过滤器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！