Logstash和Filebeat设置event.dataset值 [英] Logstash and filebeat set event.dataset value

问题描述

我有一个配置，其中filebeat从某些文件中获取日志(使用自定义格式)，然后将这些日志发送到logstash实例.

I've a configuration in which filebeat fetches logs from some files (using a custom format) and sends those logs to a logstash instance.

在logstash中，我应用了gork过滤器以拆分某些字段，然后将输出发送到我的elasticsearch实例.

In logstash I apply a gork filter in order to split some of the fields and then I send the output to my elasticsearch instance.

管道正常运行，并且已正确加载到Elasticsearch上，但不存在任何事件数据(例如 event.dataset 或 event.module ).因此，我正在寻找将这些信息添加到事件中的代码.

The pipeline works fine and it is correctly loaded on elasticsearch, but no event data is present (such as event.dataset or event.module). So I'm looking for the piece of code for adding such information to my events.

这是我的文件拍配置:

filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false

filebeat.inputs:
  - type: log
    paths:
      - /var/log/*/info.log
      - /var/log/*/warning.log
      - /var/log/*/error.log



output.logstash:
  hosts: '${ELK_HOST:logstash}:5044'

这是我的logstash管道:

Here my logstash pipeline:

input {
  beats {
    port => 5044
  }
}

filter {
    grok {
        match => { "message" => "MY PATTERN"}
    }
  mutate {
    add_field => { "logLevelLower" => "%{logLevel}" }
  }
  mutate {
    lowercase => [ "logLevelLower" ]
  }
}

output {
    elasticsearch {
    hosts => "elasticsearch:9200"
    user => "USER"
    password => "PASSWORD"
    index => "%{[@metadata][beat]}-%{logLevelLower}-%{[@metadata][version]}"
  }
}

推荐答案

您可以使用 mutate/add_field 过滤器轻松地做到这一点:

You can do it like this easily with a mutate/add_field filter:

filter {
  mutate {
    add_field => {
        "[ecs][version]" => "1.5.0"
        "[event][kind]" => "event"
        "[event][category]" => "host"
        "[event][type]" => ["info"]
        "[event][dataset]" => "module.dataset"
    }
  }
}

Elastic Common Schema文档解释了如何为 种类，类别和类型 .

The Elastic Common Schema documentation explains how to pick values for kind, category and type.

这篇关于Logstash和Filebeat设置event.dataset值的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！