在Logstash中分解json数组字符串 [英] Decompose json array string in Logstash

问题描述

我有一个RabbitMQ以JSON格式发送3个字段，Logstash rabbitmq输入插件正在使用它.

I have a RabbitMQ sending 3 fields in JSON format and it is being consumed by Logstash rabbitmq input plugin.

其中一个字段是一个JSON字符串数组，如下所示:

One of the fields is a JSON array of strings as follows:

"content": [
  "1111",
  "2222222",
  "Test 06",
  "3",
  "3232",
  "SomeValue1"
]

1. 如何将每个字符串输入字段，以便可以从可用字段中快速发现Kibana并对其进行可视化?现在，我看到带有完整字符串的内容".

1. How could I make each entry of that string into a field so that I can quickly discover and visualize in Kibana from the available fields? Right now I see "content" with that full string.

JSON数组字符串的大小根据另一个字段eventID而变化.是否可以根据eventID将字符串中的值动态映射到特定名称? 如:

The JSON array string size changes depending on another field, eventID. Would it be possible to dynamically map the values in that string to specific names depending on the eventID? Such as:

"eventID": 1,
  "content": [
  "name1": "1111",
  "name2": "2222222",
  "name3": "Test 06",
  "name4": "3",
  "name5": "3232",
  "name6": "SomeValue1"
]

"eventID": 2,
  "content": [
  "othername1": "3434",
  "othername2": "Test 10",
  "othername3": "876",
  "othername4": "Some String7"
]

我想在可用字段中使用name *和othername *. 任何帮助，将不胜感激.

I would like to have the name* and othername* in the available fields. Any help would be appreciated.

推荐答案

首先，我暂时假设您的输入已经正确解析为数组.出于测试目的，这意味着:

First, I'm going to assume for a moment that your input is properly parsed as an array already. For testing purposes, that means that this:

echo '{"eventID":1,"content":["a","b","c","d"]}' | bin/logstash -f test.conf 

其中test.conf是:

where test.conf is:

input {
    stdin { codec => json }
}
output {
    stdout { codec => rubydebug }
}

将输出以下内容:

{
       "eventID" => 1,
    "@timestamp" => 2017-08-03T19:39:13.054Z,
      "@version" => "1",
          "host" => "xxxxxx.local",
       "content" => [
        [0] "a",
        [1] "b",
        [2] "c",
        [3] "d"
    ]
}

如果是这种情况，那么您需要执行以下操作:

If that's the case, then you'll need to do something like this:

filter {
    if [eventID] == 1 {
        mutate {
            add_field => {
                "eventName" => "type1 event"
                "one0" => "%{[content][0]}"
                "one1" => "%{[content][1]}"
                "one2" => "%{[content][2]}"
                "one3" => "%{[content][3]}"
            }
            remove_field => [ "content" ]
        }
    } else if [eventID] == 2 {
        mutate {
            add_field => {
                "eventName" => "type2 event"
                "two0" => "%{[content][0]}"
                "two1" => "%{[content][1]}"
                "two2" => "%{[content][2]}"
                "two3" => "%{[content][3]}"
            }
            remove_field => [ "content" ]
        }
    }
}

这将生成这样的事件:

{
       "eventID" => 1,
    "@timestamp" => 2017-08-03T19:51:02.946Z,
      "@version" => "1",
          "host" => "xxxxxxx.local",
          "one2" => "c",
     "eventName" => "type1 event",
          "one3" => "d",
          "one0" => "a",
          "one1" => "b"
}

这篇关于在Logstash中分解json数组字符串的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！