发送给Elasticsearch之前，请向字段添加额外的值 [英] Add extra value to field before sending to elasticsearch

问题描述

我正在使用logstash，filebeat和grok将数据从日志发送到我的elastisearch实例.这是管道中的grok配置

I'm using logstash, filebeat and grok to send data from logs to my elastisearch instance. This is the grok configuration in the pipe

filter {
    grok {
        match => {
            "message" => "%{SYSLOGTIMESTAMP:messageDate} %{GREEDYDATA:messagge}"
        }
    }
}

这很好用，问题在于messageDate的格式为Jan 15 11:18:25，并且没有年份条目.
现在，我实际上知道这些文件的创建年份，并且我想知道是否可以在此过程中将值添加到字段中，也就是说，在发送给elasticsearch之前以某种方式将Jan 15 11:18:25转换为2016 Jan 15 11:18:25(显然没有编辑文件，尽管我可以轻松地做到这一点，但这只是我必须做的临时修复，而不是确定的解决方案)

This works fine, the issue is that messageDate is in this format Jan 15 11:18:25 and it doesn't have a year entry.
Now, i actually know the year these files were created in and i was wondering if it is possible to add the value to the field during the process, that is, somehow turn Jan 15 11:18:25 into 2016 Jan 15 11:18:25 before sending to elasticsearch (obviously without editing the files, which i could do and even with ease but it'll be a temporary fix to what i have to do and not a definitive solution)

如果可能，但没有运气，我曾尝试使用谷歌搜索.

I have tried googling if it was possible but no luck...

推荐答案

通过仔细阅读grok文档，我发现google无法为我找到什么，而我第一次阅读该页面时显然错过了它

By reading thoroughly the grok documentation i found what google couldn't find for me and which i apparently missed the first time i read that page

https ://www.elastic.co/guide/zh-CN/logstash/current/plugins-filters-grok.html#plugins-filters-grok-add_field

使用add_field和remove_field选项，我设法将年份添加到我的日期中，然后我使用日期插件将其发送给logstash作为时间戳.我的过滤器配置现在看起来像这样

Using the add_field and remove_field options i managed to add the year to my date, then i used the date plugin to send it to logstash as a timestamp. My filter configuration now looks like this

filter {
    grok {
        match => {
            "message" => "%{SYSLOGTIMESTAMP:tMessageDate} %{GREEDYDATA:messagge}"
            add_field => { "messageDate" => "2016 %{tMessageDate}" }
            remove_field => ["tMessageDate"]
        }
    }
    date {
        match => [ "messageDate", "YYYY MMM dd HH:mm:ss"]
    }
}

效果很好

这篇关于发送给Elasticsearch之前，请向字段添加额外的值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！