从给定的字段,根据内容,将其修改并存储在不同的新字段中 [英] From a given field, depending on the content, mofify it and store it in different new fields

查看:109
本文介绍了从给定的字段,根据内容,将其修改并存储在不同的新字段中的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我已经创建了这个新主题,以避免混淆,但与以下主题有关:已解决:



使用logstash修改字段的内容



让我们说,在代码的字段中,内容也是字,如:通知,我想创建另一个名为 code_word 的字段来存储这些字。



所以从数据库中称为代码的字段,根据其内容,它将创建 code_short code_word



我正在尝试这样的东西:

  grok {
match => {code=> (?< prefix> [a-zA-Z] +)000000%{INT:suffix}%{WORD:word}}
add_field => {code_short=> %{prefix}%{suffix}}
add_field => {code_word=> %{word}}
}

但显然不工作:_grokparsefailure。 / p>

我不知道该怎么做,我想知道是否可以使用条件如if。



这是输出:

  {
take:5,
timed_out:false,
_shards:{
total:5,
successful:5,
failed:0
},
hits:{
total:43,
max_score:1,
hits:[
{
_index test1,
_type:data,
_id:AVo3yno0PoyZlwFEpg75,
_score:1,
_source:{
@timestamp:2017-02-13T14:05:00.360Z,
code:molfix-oat,
@version:1,
tags:[]
}
},
{
_index:test1,
_ $$$$$$$$$$$$$$ 02-13T14:05:00.365Z,
code:PO0000002221,
@version:1,
tags:[]
}
},
{
_index:test1,
_type:data,
_id:AVo3yno0PoyZlwFEpg8C,
_score:1,
_source:{
@timestamp:2017-02-13T14:05:00.372Z,
code:ST0000003302
@version:1,
标签:[]
}
},
{
_index:test1 ,
_type:data,
_id:AVo3yno0PoyZlwFEpg8K,
_score:1,
_source:{
@时间戳:2017-02-13T14:05:00.382Z,
code:notifications-set,
@version:1,
tags:[]
}
},
{
_index:test1,
_type:data,
_id:AVo3yno0PoyZlwFEpg8M,
_score:1,
_source:{
@timestamp:2017-02-13T14:05:00.384Z,
code:PO0000001111,
@version:1 ,
tags:[]
}
},
{
_index:test1,
_type ,
_id:AVo3yno0PoyZlwFEpg8N,
_score:1,
_source:{
@timestamp:2017-02-13T14:05: 00.385Z,
code:PO0000000808,
@version:1,
tags:[]
}
}
{
_index:t $ 1,
_source:{
$ _ $$$$ @timestamp:2017-02-13T14:05:00.388Z,
code:TT0000000009,
@version:1,
tags :[]
}
},
{
_index:test1,
_type:data,
_id :$
_sour:$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ code:ST0000001113,
@version:1,
tags:[]
}
},
{
_index:test1,
_type:data,
_id:AVo3yno0PoyZlwFEpg8c,
_score:1,
_source :{
@timestamp:2017-02- 13T14:05:00.398Z,
code:test / bin / UT0,
@version:1,
tags:[]

},
{
_index:test1,
_type:data,
_id:AVo3yno0PoyZlwFEpg8g b $ b_score:1,
_source:{
@timestamp:2017-02-13T14:05:00.400Z,
code:PO0000001203 ,
@version:1,
标签:[]
}
}
]
}
}


解决方案

最简单的方法是检测 grokparsefailure 标记,并将值分配给 code_word 而不是:

  grok {
match => {code=> (?< prefix> [a-zA-Z] +)000000%{INT:suffix}}
add_field => {code_short=> %{prefix}%{suffix}}
}
如果[标签]中的_grokparsefailure{
mutate {
add_field => {code_word=> %{code}}
remove_field => [tags]
}
}


I have created this new topic to avoid confusion but it is related to the following topic, which was resolved:

Modify the content of a field using logstash

Let's say that in field called code the content are also words, like: notification or mountain and I would like to create another field called code_word to store those words.

So from the field called code in the database, depending on its content it would create code_short, and code_word.

I was trying something like this:

grok {
  match => { "code" => "(?<prefix>[a-zA-Z]+)000000%{INT:suffix} %{WORD:word}" }
  add_field => { "code_short" => "%{prefix}%{suffix}"}
  add_field => { "code_word" => "%{word}"}
}

but clearly is not working: _grokparsefailure.

I am not sure how to do it, I am wondering if it could be done using a conditional like an "if".

This is the output:

    {
   "took": 5,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 43,
      "max_score": 1,
      "hits": [
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg75",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.360Z",
               "code": "molfix-oat",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg77",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.365Z",
               "code": "PO0000002221",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8C",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.372Z",
               "code": "ST0000003302",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8K",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.382Z",
               "code": "notifications-set",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8M",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.384Z",
               "code": "PO0000001111",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8N",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.385Z",
               "code": "PO0000000808",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8R",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.388Z",
               "code": "TT0000000009",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8U",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.391Z",
               "code": "ST0000001113",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8c",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.398Z",
               "code": "test/bin/UT0",
               "@version": "1",
               "tags": []
            }
         },
         {
            "_index": "test1",
            "_type": "data",
            "_id": "AVo3yno0PoyZlwFEpg8g",
            "_score": 1,
            "_source": {
               "@timestamp": "2017-02-13T14:05:00.400Z",
               "code": "PO0000001203",
               "@version": "1",
               "tags": []
            }
         }
      ]
   }
}

解决方案

The easiest way to go about it is to detect the grokparsefailure tag and assign the value to code_word instead:

grok {
  match => { "code" => "(?<prefix>[a-zA-Z]+)000000%{INT:suffix}" }
  add_field => { "code_short" => "%{prefix}%{suffix}"}
}
if "_grokparsefailure" in [tags] {
   mutate {
      add_field => { "code_word" => "%{code}"}
      remove_field => ["tags"]
   }
}

这篇关于从给定的字段,根据内容,将其修改并存储在不同的新字段中的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆