弹性搜索批量API，管道和地理IP [英] Elastic Search Bulk API, Pipeline and Geo IP

问题描述

  {index：{_ index： w，_ type：日志，_ id：325d05bb6900440e}} 
 {id：325d05bb6900440e，country：US，ip ，协议：HTTP / 1.1，方法：GET，主机：xxxxx，user_agent：Mozilla / 5.0（Windows NT 10.0; WOW64）AppleWebKit / 537.36（KHTML， ）Chrome / 54.0.2840.99 Safari / 537.36，uri：/？a =>< script> alert（1）< / script>，request_duration：1999872，triggered_rule_ids：[100030 ]， 行动 ： 挑战  cloudflare_location： 未知， occurred_at： 2017-01-23T17：38：58.46Z， rule_detail：[{ ID： ，描述：ARGS：A}]，rule_message：Generic XSS Probing，type：waf，rule_id：100030} 
  pre> 
 
 
我有一个ip在数据中，我想使用GEOIP插件转入经度和纬度。
 
 
 
我创建了一个pipleine：
  PUT _ingest / pipeline / geoip-info 
 {
 description：添加地理位置信息，
游标：[
 {
geoip：{
field：ip，
target_field：client_geoip，
properties [location]，
ignore_failure：true 
} 
} 
] 
}`
  
但是，当导入数据时，管道被忽略可以有人解释我如何修改批量API以通过管道传递信息，以便为我添加long和lat创建地图。
 
 
 
谢谢
解决方案
没有管道名称
  here 
 | 
 V 
 {index：{_ index：waf，_ type：logs，_ id：325d05bb6900440e，pipeline：geoip-info}} 
 {id：325d05bb6900440e，country：US，ip：1.1.1.1，protocol：HTTP / 1.1主机：xxxxx，user_agent：Mozilla / 5.0（Windows NT 10.0; WOW64）AppleWebKit / 537.36（KHTML，像Gecko）Chrome / 54.0.2840.99 Safari / 537.36，uri：/？a = ><脚本>警报（1） - ; /脚本> 中， request_duration：1999872， triggered_rule_ids：[ 100030]， 行动： 挑战， cloudflare_location： 未知， __，rule_message：Generic XSS Probing（通用XSS探测） ，type：waf，rule_id：100030} 
  
或者你也可以在大量网址中设置
  POST _bulk？pipeline = geoip-info 
  / pre> 
I import data to my ELK stack using the Bulk API.
{"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e"}}
{"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}
I have an ip in the data that i want to turn in to longitude and latitude using the GEOIP addon.

I have created a pipleine:
PUT _ingest/pipeline/geoip-info
{
"description": "Add geoip info",
"processors": [
{
  "geoip": {
    "field": "ip",
    "target_field": "client_geoip",
    "properties": ["location"],
    "ignore_failure": true
  }
}
]
}`
However when I import the data the pipeline is ignored can someone explain how i modify the bulk API to pass the information through a pipeline in order to add long and lat for me to create maps.

Thanks
解决方案
In your bulk call you're missing the pipeline name
                                                                    here
                                                                      |
                                                                      V
{"index":{"_index":"waf","_type":"logs","_id":"325d05bb6900440e", "pipeline": "geoip-info"}}
{"id":"325d05bb6900440e","country":"US","ip":"1.1.1.1","protocol":"HTTP/1.1","method":"GET","host":"xxxxx","user_agent":"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36","uri":"/?a=><script>alert(1)</script>","request_duration":1999872,"triggered_rule_ids":["100030"],"action":"challenge","cloudflare_location":"unknown","occurred_at":"2017-01-23T17:38:58.46Z","rule_detail":[{"id":"","description":"ARGS:A"}],"rule_message":"Generic XSS Probing","type":"waf","rule_id":"100030"}
Or you can also set it in the bulk URL
POST _bulk?pipeline=geoip-info


                        
这篇关于弹性搜索批量API，管道和地理IP的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！