Grok自定义模式中的Lookahead和Lookbehind语法 [英] Syntax for Lookahead and Lookbehind in Grok Custom Pattern

问题描述

我正在尝试在Grok自定义模式中使用后向和前瞻，并在无法解决的Grok调试器中获取模式匹配错误.

I'm trying to use a lookbehind and a lookahead in a Grok custom pattern and getting pattern match errors in the Grok debugger that I cannot resolve.

这是用于归档系统日志.我目前正在尝试解析postgrey应用程序.

This is for archiving system logs. I am currently trying to parse the postgrey application.

给出如下数据:

2019-04-09T11:41:31-05:00 67.157.192.7 postgrey: action=pass, reason=triplet found, delay=388, client_name=unknown, client_address=103.255.78.9, sender=[e-mail], recipient=[e-mail]

我正在尝试使用以下内容将"action =和紧随其后的逗号之间的字符串作为字段"postgrey_action":

I'm trying to use the following to pull the string between "action=" and the comma immediately following it as the field "postgrey_action":

%{TIMESTAMP_ISO8601:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG} (?<postgrey_action>(?<=action=).+?(?=\,))

我希望看到以下输出:

{
  "program": "dhcpd:",
  "logsource": "66.146.192.67",
  "timestamp": "2019-04-09T11:41:31-05:00"
  "postgrey_action": "pass"
}

相反，我从调试器收到提供的Grok模式与输入中的数据不匹配".

Instead, from the debugger, I receive "Provided Grok patterns do not match data in the input".

我该如何正确地进行这种先行/后行工作?

How can I properly make this lookbehind/lookahead work?

我应该注意，如果在Grok模式的末尾没有postgrey_action匹配项，Grok调试器将按预期运行并运行(使用linux-syslog和grok-patterns).

I should note that without the postgrey_action match at the end of the Grok pattern, the Grok Debugger runs and works as expected (using linux-syslog and grok-patterns).

Logstash版本6.3.2

Logstash version 6.3.2

推荐答案

作为一种变通办法，我不得不使用自定义模式文件修改语法，并在每个过滤器中使用patterns_dir指令对其进行引用.

As a work around, I have resorted to modifying my syntax, using a custom patterns file, and referencing it in each filter using the patterns_dir directive.

例如. 我的模式:

POSTGREY %{TIMESTAMP_ISO8601:timestamp} (?:%{SYSLOGFACILITY} )?%{SYSLOGHOST:logsource} %{SYSLOGPROG} (action=)%{WORD:postgrey_action}(,) (reason=)%{DATA:postgrey_reason}(,) (delay=)%{NUMBER:postgrey_delay}(,) (client_name=)%{IPORHOST}(,) (client_address=)%{IPORHOST:postgrey_clientaddr}(,) (sender=)%{EMAILADDRESS:postgrey_sender}(,)

我的过滤器:

    if "postgrey" in [program]  {
        grok {
        match => { "message" => "%{POSTGREY}"}
        patterns_dir => ["/etc/logstash/patterns"]
        overwrite => [ "message" ]
        }
    }

但是，此替代方法仍然无法回答我的原始问题，即为什么我的初始方法不起作用?

查看 Oniguruma Regex文档和

Looking at the Oniguruma Regex documentation and the Grok filters documentation, it's not clear to me what is wrong with my original syntax or how a look-ahead/look-behind should be properly implemented with grok regex named capture. If it is not supported, it should not be documented as such.

这篇关于Grok自定义模式中的Lookahead和Lookbehind语法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！