如何从文本文件行解析 logstash/grok 中的 json? [英] How to parse json in logstash /grok from a text file line?

问题描述

我有一个看起来像这样的日志文件(简化)

I have a logfile which looks like this ( simplified)

Logline 示例

MyLine data={"firstname":"bob","lastname":"the builder"}

我想提取 data 中包含的 json 并创建两个字段，一个用于名字，一个用于最后.但是，我得到的输出是这样的:

I'd like to extract the json contained in data and create two fields, one for firstname, one for last. However, the ouput i get is this:

{"message":"Line data={"firstname":"bob","lastname":"the builder"}
","@version":"1","@timestamp":"2015-11-26T11:38:56.700Z","host":"xxx","path":"C:/logstashold/bin/input.txt","MyWord":"Line","parsedJson":{"firstname":"bob","lastname":"the builder"}}

如你所见

..."parsedJson":{"firstname":"bob","lastname":"the builder"}}

这不是我需要的，我需要在 kibana 中为名字和姓氏创建字段，但 logstash 没有使用 json 过滤器提取字段.

That's not what I need, I need to create fields for firstname and lastname in kibana, but logstash isn't extracting the fields out with the json filter.

LogStash 配置

input {
  file {
        path => "C:/logstashold/bin/input.txt"        
       }
}

filter {     

   grok {
            match => { "message" => "%{WORD:MyWord} data=%{GREEDYDATA:request}"}        
        }   

    json{
        source => "request"
        target => "parsedJson"
        remove_field=>["request"]
    }   
}   

output {  
    file{
        path => "C:/logstashold/bin/output.txt"
    }   
}

非常感谢任何帮助，我确定我错过了一些简单的东西

Any help greatly appreciated, I'm sure I'm missing out something simple

谢谢

推荐答案

在您的 json 过滤器之后添加另一个名为 mutate 以添加您将采用的两个字段来自 parsedJson 字段.

After your json filter add another one called mutate in order to add the two fields that you would take from the parsedJson field.

filter {
  ...
  json {
     ...
  }
  mutate {
    add_field => {
      "firstname" => "%{[parsedJson][firstname]}"
      "lastname" => "%{[parsedJson][lastname]}"
    }
  }
}

对于上面的示例日志行，将给出:

For your sample log line above that would give:

{
       "message" => "MyLine data={"firstname":"bob","lastname":"the builder"}",
      "@version" => "1",
    "@timestamp" => "2015-11-26T11:54:52.556Z",
          "host" => "iMac.local",
        "MyWord" => "MyLine",
    "parsedJson" => {
        "firstname" => "bob",
         "lastname" => "the builder"
    },
     "firstname" => "bob",
      "lastname" => "the builder"
}

这篇关于如何从文本文件行解析 logstash/grok 中的 json?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！