如何在Elasticsearch提示器中使用Logstash导入CSV数据以完成字段类型 [英] How to import CSV data using Logstash for field type completion in Elasticsearch suggester

问题描述

ElasticSearch索引创建

ElasticSearch Index Creation

curl -XPOST 'http://localhost:9200/music/' -d '{}'

字段映射

curl -XPUT 'http://localhost:9200/music/_mapping/song' -d '
{
  "properties": {
    "name" : {
      "type" : "string"
    },
    "suggest": {
      "type" : "completion"
    }
  }
}'

LogStash配置文件musicStash.config

LogStash config file, musicStash.config

input {
    file {
        path => "pathToCsv"
        start_position => beginning
    }
}

filter {  
    csv {
        columns => ["id", "name", "suggest"]
        separator => ","
    }
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_id => "%{id}"
    }
}

现在在执行logstash配置文件时，在elasticsearch控制台中收到以下异常

Now while executing logstash config file, received following exception in elasticsearch console

failed to put mappings on indices [[music]], type [logs]
java.lang.IllegalArgumentException: Mapper for [suggest] conflicts with existing mapping in other types:
[mapper [suggest] cannot be changed from type [completion] to [string]]
at org.elasticsearch.index.mapper.FieldTypeLookup.checkCompatibility(FieldTypeLookup.java:117)

并且在logstash控制台中收到错误，

And error received in logstash console,

response=>{"index"=>{"_index"=>"music", "_type"=>"logs", "_id"=>"5", "status"=>400, 
"error"=>{"type"=>"illegal_argument_exception", 
"reason"=>"Mapper for [suggest] conflicts with existing mapping in other types:\n[mapper [suggest] cannot be changed from type [completion] to [string]]"}}}, :level=>:warn}

那么如何通过Logstash导入csv文件来实现elasticsearch自动完成功能.

So how to achieve elasticsearch auto-complete feature by importing csv file through Logstash.

推荐答案

您在elasticsearch输出中缺少以下设置:

You're missing the following setting in your elasticsearch output:

document_type => "song"

发生的事情是logstash正在创建一个名为logs的新类型(

What happens is that logstash is creating a new type called logs (by default) and since as of ES 2.0 it is forbidden to have two fields with the same name but different types (string vs completion) in the same index, it's erroring out.

只需像这样修改您的输出，它将起作用:

Just modify your output like this and it will work:

output {
    elasticsearch {
        hosts => "localhost"
        index => "music"
        document_type => "song"
        document_id => "%{id}"
    }
}

这篇关于如何在Elasticsearch提示器中使用Logstash导入CSV数据以完成字段类型的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！