Logstash grok过滤器来标记接收和退回的消息 [英] Logstash grok filter to tag received and bounced messages

问题描述

Sthg让我疯狂，我想解析Postfix日志以了解电子邮件的状态，这里是我迄今为止所尝试的：

Sthg makes me crazy, I would like to parse Postfix logs to know the status of emails, here is what I tried so far :

input {
   file {path => "/var/log/mail.log"}
}

filter {
    kv {
        trim => "<>"
    }

    if [message] =~ /[ "status=bounced" ]/ {
        grok {
            patterns_dir => "/etc/logstash/patterns"
            match => {"message" => "%{SYSLOGBASE} (?<QID>[0-9A-F]{10}): %{GREEDYDATA:message}"}
            add_tag => "bounce"
        }
    }

}
output {
   if "bounce" in [tags] {
      stdout { codec => rubydebug }
   }
}

---

mail.log示例：

Jul 26 04:18:34 mx12 postfix / cleanup [20659]： 3mfHGL1r9gzyQP ： message-id =< [e-mail]>

Jul 26 04:18:34 mx12 postfix/cleanup[20659]: 3mfHGL1r9gzyQP: message-id=<[e-mail]>

Jul 26 04:18:34 mx12 postfix / smtp [20662]： 3mfHGL1r9gzyQP ：to =，relay = 127.0.0.2 [127.0.0.2]：25，延迟= 0.53，延迟= 0.13 / 0 / 0.23 / 0.16，dsn = 2.0.0， 状态=已发送 / 退回

Jul 26 04:18:34 mx12 postfix/smtp[20662]: 3mfHGL1r9gzyQP: to=, relay=127.0.0.2[127.0.0.2]:25, delay=0.53, delays=0.13/0/0.23/0.16, dsn=2.0.0, status=sent / bounced

：

我发送电子邮件到现有的电子邮件地址，mail.log中的状态是：

I send an email to an existing email address, the status in mail.log is :

sent（250 ok）：OKAY

sent (250 ok) : OKAY

但是这里是Logstash所说的：

But here is what Logstash tells :

..和我看到每个postfix程序生成的每个消息（qmgr，smtp，qmgr再次..）。换句话说，对于所有甚至不包含status = bounced的消息，

.. and I see that for every message generated by each postfix program (qmgr, smtp, qmgr again..). In other words, for all messages that even not contain "status=bounced".

然后我也尝试过：

   if [message] =~ /[ "bounced" ]/ {
     mutate {add_tag => [ "bounce" ]}
  }

  if [message] =~ /[ "message-id", "(.*)\@www\.mydomain\.fr" ]/ {
      mutate { add_tag => [ "send" ] }
  }
  grok {
       match => {"message" => "%{SYSLOGBASE} (?<QID>[0-9A-F]{10}): %{GREEDYDATA:message}"}
  }

结果2：
Logstash总是添加2个标签：bounce + send：（

Result 2 : Logstash add here always 2 tags : bounce + send :(

预期结果

我尝试做的是完全是这个配置文件，但它是使用旧版本的Logstash （例如，grep现在不可用），但这正是我尝试使工作：

What I try to do is exactly this config file, but it was made with an old version of Logstash ("grep" for example is not available now), but this is exactly what I try to make working :

http://tales.itnobody.com/2013/07/using-logstash-to-log-smtp-bounces-like-a-boss.html

一个字：

1. 任何带有DSN的条目 - 记录：QID，dsn


2. 匹配message-id =< hashRegex>的任何条目 - RECORD：QID，message-id

如下：

    output{
       if "bounce" in [tags] {
           exec {
                command => "php -f /path/LogDSN.php %{QID} %{dsn} &"
           }
       }

       if "send" in [tags] {
           exec {
                command => "php -f /path/LogOutbound.php %{QID} %{message-id} &"
           }
       }
     }

但是有一个问题我的过滤器，让我疯狂，

But there is a problem in my filter, that makes me crazy,

任何想法？

推荐答案

p>我发现了这个问题。

I have found the problem.

来自这个测试：

if [message] =~ /[ "bounced" ]/ {
  mutate {add_tag => [ "bounce" ]}
}

正则表达式是 / ，所以你的正则表达式是这样评估的：
https ：//regex101.com/r/eaB5jp/2

The regex is the part between the /, so your regex is evaluated like that : https://regex101.com/r/eaB5jp/2

所以所有的行将匹配并获取标签。

So all your lines will match and get the tag.

为了工作，测试应该是：

In order to work, the test should be:

if [message] =~ /bounced/ {
  mutate {add_tag => [ "bounce" ]}
}

这篇关于Logstash grok过滤器来标记接收和退回的消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！