Logstash：从数组到字符串的XML到JSON的输出 [英] Logstash: XML to JSON output from array to string

问题描述

我正在尝试使用Logstash将XML转换为ElasticSearch的JSON。我可以将读取的值发送到ElasticSearch。问题是所有的值都作为数组出来。我想让它们像弦一样出来。我知道我可以单独为每个字段执行替换，但是我遇到一个问题，嵌套字段是3级深。

I am in the process of trying to use Logstash to convert an XML into JSON for ElasticSearch. I am able to get the the values read and sent to ElasticSearch. The issue is that all the values come out as arrays. I would like to make them come out as just strings. I know I can do a replace for each field individually, but then I run into an issue with nested fields being 3 levels deep.

XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<acs2:SubmitTestResult xmlns:acs2="http://tempuri.org/" xmlns:acs="http://schemas.sompleace.org" xmlns:acs1="http://schemas.someplace.org">
    <acs2:locationId>Location Id</acs2:locationId>
    <acs2:userId>User Id</acs2:userId>
    <acs2:TestResult>
        <acs1:CreatedBy>My Name</acs1:CreatedBy>
        <acs1:CreatedDate>2015-08-07</acs1:CreatedDate>
        <acs1:Output>10.5</acs1:Output>
    </acs2:TestResult>
</acs2:SubmitTestResult>

Logstash Config

input {
    file {
        path => "/var/log/logstash/test.xml"
    }
}
filter {
    multiline {
        pattern => "^\s\s(\s\s|\<\/acs2:SubmitTestResult\>)"
        what => "previous"
    }
    if "multiline" in [tags] {
        mutate {
            replace => ["message", '<?xml version="1.0" encoding="UTF-8" standalone="yes"?>%{message}']
        }
        xml {
            target => "SubmitTestResult"
            source => "message"
        }
        mutate {
            remove_field => ["message", "@version", "host", "@timestamp", "path", "tags", "type"]
            remove_field => ["entry", "[SubmitTestResult][xmlns:acs2]", "[SubmitTestResult][xmlns:acs]", "[SubmitTestResult][xmlns:acs1]"]

            # This works
            replace => [ "[SubmitTestResult][locationId]", "%{[SubmitTestResult][locationId]}" ]

            # This does NOT work
            replace => [ "[SubmitTestResult][TestResult][CreatedBy]", "%{[SubmitTestResult][TestResult][CreatedBy]}" ]
        }
    }
}
output {
    stdout {
        codec => "rubydebug"
    }
    elasticsearch {
        index => "xmltest"
        cluster => "logstash"
    }
}

示例输出

{
   "_index": "xmltest",
   "_type": "logs",
   "_id": "AU8IZBURkkRvuur_3YDA",
   "_version": 1,
   "found": true,
   "_source": {
      "SubmitTestResult": {
         "locationId": "Location Id",
         "userId": [
            "User Id"
         ],
         "TestResult": [
            {
               "CreatedBy": [
                  "My Name"
               ],
               "CreatedDate": [
                  "2015-08-07"
               ],
               "Output": [
                  "10.5"
               ]
            }
         ]
      }
    }
}

如您所见，输出是每个元素的数组（除了替换的locationId除外）。我试图不必为每个元素替换。有没有办法调整配置使输出正确放置？如果没有，我如何在替换中获得3级深度？

As you can see, the output is an array for each element (except for the locationId I replaced with). I am trying to not have to do the replace for each element. Is there a way to adjust the config to make the output come put properly? If not, how do I get 3 levels deep in the replace?

- 更新 -

我想到了如何进入测试结果的第三级。替换为：

I figured out how to get to the 3rd level in Test Results. The replace is:

replace => [ "[SubmitTestResult][TestResult][0][CreatedBy]", "%{[SubmitTestResult][TestResult][0][CreatedBy]}" ]

推荐答案

我想出来了。这是解决方案。

I figured it out. Here is the solution.

replace => [ "[SubmitTestResult][TestResult][0][CreatedBy]", "%{[SubmitTestResult][TestResult][0][CreatedBy]}" ]

这篇关于Logstash：从数组到字符串的XML到JSON的输出的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！