Kibana 4.0.1中的GeoHash聚合下不存在“平铺地图geo.location"字段 [英] Tile Map geo.location field not present under GeoHash aggregation in Kibana 4.0.1

问题描述

我正在尝试在Kibana 4.0.1中使用切片地图可视化.

I'm trying to get a tile map visualization working in Kibana 4.0.1.

我可以在发现"部分中看到geoip.location数据，但是当我将其可视化为平铺地图"时，选择Geo Coordinates存储桶类型，然后选择聚合类型GeoHash，然后再选择Field它是空白.

I can see the geoip.location data in the Discover section but when I go to Visualize it as a Tile Map, I select Geo Coordinates bucket type, then aggregation type GeoHash, then when I go to Field it's blank.

• 基巴纳语4.0.1
• Logstash 1.4.2-1-2c0f5a1
• Elasticsearch 1.4.4
• 在Debian 7 64bit上运行

这是我的logstash配置:

Here's my logstash config:

    input {
    udp {
      port => 5514
      type => cisco
    }
}
filter {
  grok {
    match => { "message" => "\<%{NUMBER:num}\>%{NUMBER:seq}: \*%{SYSLOGTIMESTAMP:date}: \%SEC-6-IPACCESSLOGP: list Internet\-In\-%{WORD:acl_ver} denied %{WORD:protocol} %{IP:src}\(%{NUMBER:sport}\) \-\> %{IP:dest}\(%{NUMBER:dport}\), %{INT:hitcnt}" }
    add_tag => ["grok_match", "cisco_acl_message"]
    remove_field => [ "message" ]
  }
  geoip {
   source => "src"
   target => "geoip"
   add_tag => ["geoip"]
   database => "/etc/logstash/GeoLiteCity.dat"
 }
}
output {
  elasticsearch { 
    host => localhost
    index => [ "firewall-%{+YYYY.MM.DD}" ]
  }
}

这是一个日志记录的示例:

And this is an example of a log record:

{
  "_index": "firewall-2015.04.105",
  "_type": "cisco",
  "_id": "dJhGF6RtQuGXtlBTRCu2mQ",
  "_score": null,
  "_source": {
    "@version": "1",
    "@timestamp": "2015-04-15T21:06:08.357Z",
    "type": "cisco",
    "host": "172.17.10.1",
    "num": "190",
    "seq": "1872",
    "date": "Apr 15 21:08:05.878",
    "acl_ver": "20150223",
    "protocol": "tcp",
    "src": "94.102.51.96",
    "sport": "26820",
    "dest": "12.34.56.78",
    "dport": "5900",
    "hitcnt": "1",
    "tags": [
      "grok_match",
      "cisco_acl_message",
      "geoip",
      "_grokparsefailure",
      "geoip"
    ],
    "geoip": {
      "ip": "94.102.51.96",
      "country_code2": "NL",
      "country_code3": "NLD",
      "country_name": "Netherlands",
      "continent_code": "EU",
      "region_name": "07",
      "city_name": "Amsterdam",
      "postal_code": "1000",
      "latitude": 52.349999999999994,
      "longitude": 4.916699999999992,
      "timezone": "Europe/Amsterdam",
      "real_region_name": "Noord-Holland",
      "location": [
        4.916699999999992,
        52.349999999999994
      ],
      "coordinates": [
        4.916699999999992,
        52.349999999999994
      ]
    }
  },
  "fields": {
    "@timestamp": [
      1429131968357
    ]
  },
  "sort": [
    1429131968357
  ]
}

有什么想法我想念的吗?

Any ideas what I'm missing?

答案

在阿兰指出我正确的方向之后，我开始研究场映射.这是我的处理方法:

After Alain pointed me in the right direction I started looking at field mappings. Here's how I went about it:

首先，我检查了geoip字段类型(我的索引称为firewall*)

First I checked the geoip field type (my index is called firewall*)

curl http://localhost:9200/firewall*/_mapping/cisco/field/geoip.location?pretty

这又回来了:

{
  "firewall-2015.04.107" : {
    "mappings" : {
      "cisco" : {
        "geoip.location" : {
          "full_name" : "geoip.location",
          "mapping":{"location":{"type":"float"}}
        }
      }
    }
  }
}

float类型的位置是为什么我无法添加平铺地图可视化的原因.我需要将其更改为geo_point.

The float type location is why I can't add a tile map visualization. I need to change this to geo_point.

经过大量的挖掘，我发现将location类型的映射更改为geo_point的一种方法是使用输出模板.我从

After loads of digging I found one way to change the mapping of the location type to geo_point was to use an output template. I copied the default elasticsearch_templte.json file from

/opt/logstash/lib/logstash/outputs/elasticsearch/elasticsearch-template.json

到

   /etc/logstash/templates/elasticsearch-firewall.json

编辑它，并将模板从logstash*更改为firewall*(或任何您的索引名称模式) "template" : "firewall*"，

Edit it and change the template from logstash* to firewall* (or whatever your index name pattern is) "template" : "firewall*",

编辑logstash配置文件，并将输出更改为此:

Edit the logstash config file and change the output to this:

output {
  elasticsearch { 
    host => localhost
    index => [ "firewall-%{+YYYY.MM.DD}" ]
    template => "/etc/logstash/templates/elasticsearch-firewall.json"
    template_name => "firewall"

  }
}

删除firewall*索引.

警告:这将删除您所有现有的可搜索数据！如果此数据很重要，则需要找到一种无需删除索引即可动态更改字段类型的方法.我在Google搜索中看到过提及此消息的消息，所以我知道这是可能的.

curl -XDELETE http://localhost:9200/firewall*

此后，我重新启动了logstash和elasticsearch.

I restarted logstash and elasticsearch after this.

一旦我再次开始接收日志，我便再次检查了映射

Once I started receiving logs again I checked the mapping again

 curl http://localhost:9200/firewall*/_mapping/cisco/field/geoip.location?pretty
{
  "firewall-2015.04.107" : {
    "mappings" : {
      "cisco" : {
        "geoip.location" : {
          "full_name" : "geoip.location",
          "mapping":{"location":{"type":"geo_point"}}
        }
      }
    }
  }
}

看看现在的位置类型是geo_point:-)

See how the location type is now geo_point :-)

现在，我可以添加一个Tile Map可视化.

Now I can add a Tile Map visualization.

推荐答案

Kibana正在使用字段的映射来确定它是否是geo_point，因此可以在地图中使用.

Kibana is using the mapping of the field to determine if it's a geo_point and therefore can be used in the map.

因此，您应该检查映射并将字段更改为geo_point.

So, you should check your mapping and change the field to a geo_point.

请参见文档.

这篇关于Kibana 4.0.1中的GeoHash聚合下不存在“平铺地图geo.location"字段的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！