Logstash未在弹性搜索中创建索引 [英] Logstash is not creating index in elastic search

问题描述

在控制台上未显示任何错误.这是我正在运行的命令(在Windows 10上)-

It doesn't show any error on console. Here is the command I'm running ( On Windows 10. ) -

logstash --verbose -f logstash-sample.conf

这是我的logstash-sample.conf文件-

input {
  file {
    path => "C:\Users\17739\Documents\IIT\CSP586\tutorial\project\ChicagoSocialHub\backend-build-divvy-status\divvy_stations_status.csv"
    start_position => "beginning"
  }
}

filter {
   csv{
      separator => ","
      columns => ["altitude", "availableBikes", "availableDocks", "city", "id", "is_renting", "kioskType", "landMark", "lastCommunicationTime", "latitude", "location", "longitude", "postalCode", "stAddress1", "stAddress2", "stationName", "status", "statusKey", "statusValue", "testStation", "totalDocks"]
      }
}
output {
  elasticsearch { 
  hosts => ["localhost:9200"]
  index => "divvy_stations_status"
  document_type => "status"
   }
  stdout {
  codec => rubydebug
  }
}

这是Logstash控制台的输出-

And here is the logstash console output -

C:\Users\17739\Documents\IIT\CSP586\logstash-6.6.2\bin>logstash --verbose -f logstash-sample.conf
Sending Logstash logs to C:/Users/17739/Documents/IIT/CSP586/logstash-6.6.2/logs which is now configured via log4j2.properties
[2019-03-17T12:56:36,728][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-03-17T12:56:36,745][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.6.2"}
[2019-03-17T12:56:41,603][WARN ][logstash.outputs.elasticsearch] You are using a deprecated config setting "document_type" set in elasticsearch. Deprecated settings will continue to work, but are scheduled for removal from logstash in the future. Document types are being deprecated in Elasticsearch 6.0, and removed entirely in 7.0. You should avoid this feature If you have any questions about this, please visit the #logstash channel on freenode irc. {:name=>"document_type", :plugin=><LogStash::Outputs::ElasticSearch index=>"divvy_stations_status", id=>"f84c43181aab6f7bf9e89c0412ada5b5ead116534f6661194800152751a28e87", hosts=>[//localhost:9200], document_type=>"status", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_1264be19-323c-4896-8214-929f15a74251", enable_metric=>true, charset=>"UTF-8">, workers=>1, manage_template=>true, template_name=>"logstash", template_overwrite=>false, doc_as_upsert=>false, script_type=>"inline", script_lang=>"painless", script_var_name=>"event", scripted_upsert=>false, retry_initial_interval=>2, retry_max_interval=>64, retry_on_conflict=>1, ilm_enabled=>false, ilm_rollover_alias=>"logstash", ilm_pattern=>"{now/d}-000001", ilm_policy=>"logstash-policy", action=>"index", ssl_certificate_verification=>true, sniffing=>false, sniffing_delay=>5, timeout=>60, pool_max=>1000, pool_max_per_route=>100, resurrect_delay=>5, validate_after_inactivity=>10000, http_compression=>false>}
[2019-03-17T12:56:43,234][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2019-03-17T12:56:43,548][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}}
[2019-03-17T12:56:43,695][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"}
[2019-03-17T12:56:43,735][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>6}
[2019-03-17T12:56:43,739][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2019-03-17T12:56:43,768][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//localhost:9200"]}
[2019-03-17T12:56:43,782][INFO ][logstash.outputs.elasticsearch] Using mapping template from {:path=>nil}
[2019-03-17T12:56:43,801][INFO ][logstash.outputs.elasticsearch] Attempting to install template {:manage_template=>{"template"=>"logstash-*", "version"=>60001, "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"dynamic_templates"=>[{"message_field"=>{"path_match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"text", "norms"=>false, "fields"=>{"keyword"=>{"type"=>"keyword", "ignore_above"=>256}}}}}], "properties"=>{"@timestamp"=>{"type"=>"date"}, "@version"=>{"type"=>"keyword"}, "geoip"=>{"dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip"}, "location"=>{"type"=>"geo_point"}, "latitude"=>{"type"=>"half_float"}, "longitude"=>{"type"=>"half_float"}}}}}}}}
[2019-03-17T12:56:44,323][INFO ][logstash.inputs.file     ] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"C:/Users/17739/Documents/IIT/CSP586/logstash-6.6.2/data/plugins/inputs/file/.sincedb_6f34c293ff88e0ad3c31e4a0f32e43d9", :path=>["C:\\Users\\17739\\Documents\\IIT\\CSP586\\tutorial\\project\\ChicagoSocialHub\\backend-build-divvy-status\\divvy_stations_status.csv"]}
[2019-03-17T12:56:44,369][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0x2e991954 run>"}
[2019-03-17T12:56:44,440][INFO ][filewatch.observingtail  ] START, creating Discoverer, Watch with file and sincedb collections
[2019-03-17T12:56:44,441][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-03-17T12:56:44,786][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}

运行logstash --verbose -f logstash-sample.conf命令后，我希望此新索引divvy_stations_status在弹性搜索索引中可见(也已通过kibana选中，也未显示在此处).这是一个真实的期望吗?这是我的 输出-

After running logstash --verbose -f logstash-sample.conf command , My expectation is this new index divvy_stations_status will be visible in elastic search indices(checked with kibana as well , not showing up there as well) . Is that a true expectation ? Here is my output -

推荐答案

解决方案是在文件路径中提供forward slash /而不是backward slash \.这很奇怪，因为在Windows中，我们始终对路径分隔符使用正斜杠.而且更怪异的是，它从未引发过诸如找不到文件或其他内容"之类的错误

The solution was giving forward slash / instead of backward slash \ in the file path. This was weird, since in windows we always use a forward slash for a path separator. And what more weird was it never threw an error like 'File not found or something'

因此在logstash-sample.conf文件中使用以下命令-

So in logstash-sample.conf file use the following -

path => "C:/Users/17739/Documents/IIT/CSP586/tutorial/project/ChicagoSocialHub/backend-build-divvy-status/divvy_stations_status.csv" 

这篇关于Logstash未在弹性搜索中创建索引的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！