Logstash:根据某些条件重命名嵌套字段 [英] Logstash: Renaming nested fields based on some condition

问题描述

我试图在迁移到Amazonelasticsearch时重命名Elasticsearch中的嵌套字段

I am trying to rename the nested fields from Elasticsearch while migrating to Amazonelasticsearch

在文档中，我想更改

1.如果 value 字段具有JSON类型.将值字段更改为值关键字，并删除"value-whitespace"和"value-standard"(如果存在)

1.If the value field has JSON type. Change the value field to value-keyword and remove "value-whitespace" and "value-standard" if present

2.如果value字段的大小大于15.将 value 字段更改为 value-standard

2.If the value field has a size of more than 15. Change the value field to value-standard

 "_source": {
          "applicationid" : "appid",
          "interactionId": "716bf006-7280-44ea-a52f-c79da36af1c5",
          "interactionInfo": [
            {
              "value": """{"edited":false}""",
              "value-standard": """{"edited":false}""",
              "value-whitespace" :  """{"edited":false}"""
              "title": "msgMeta"
            },
            {
              "title": "msg",
              "value": "hello testing",
            },
            {
              "title": "testing",
              "value": "I have a text that can be done and changed only the size exist more than 20 so we applied value-standard ",
            }
          ],
          "uniqueIdentifier": "a21ed89c-b634-4c7f-ca2c-8be6f31ae7b3",
        }
      }

最终结果应该是

 "_source": {
          "applicationid" : "appid",
          "interactionId": "716bf006-7280-44ea-a52f-c79da36af1c5",
          "interactionInfo": [
            {
              "value-keyword": """{"edited":false}""",
              "title": "msgMeta"
            },
            {
              "title": "msg",
              "value": "hello testing",
            },
            {
              "title": "testing",
              "value-standard": "I have a text that can be done and changed only the size exist more than 20 and so we applied value-standard  ",
            }
          ],
          "uniqueIdentifier": "a21ed89c-b634-4c7f-ca2c-8be6f31ae7b3",
        }
      }

推荐答案

为此创建了解决方案.我在Logstash中使用了ruby过滤器来检查每个文档以及嵌套文档 这是红宝石代码

Founded the solution for this one. I have used a ruby filter in Logstash to check each and every document as well as nested document Here is the ruby code

require 'json'

def register(param)
end

def filter(event)
  infoarray = event.get("interactionInfo")
  infoarray.each {  |x|
      if x.include?"value"
         value = x["value"]
         if value.length > 15
           apply_only_keyword(x)
         end
       end
      if x.include?"value"
        value = x["value"]
         if validate_json(value)
           apply_only_keyword(x)
         end
       end
  }
event.set("interactionInfo",infoarray)
return [event]
end


def validate_json(value)
  if value.nil?
    return false
  end
  JSON.parse(value)
  return true
rescue JSON::ParserError => e
  return false
end

def apply_only_keyword(x)
  x["value-keyword"] = x["value"]
  x.delete("value")
  if x.include?"value-standard"
    x.delete("value-standard")
  end
  if x.include?"value-whitespace"
    x.delete("value-whitespace")
  end
end

这篇关于Logstash:根据某些条件重命名嵌套字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！