Logstash 无法从文本文件中读取 json [英] Logstash unable to read json from text file
本文介绍了Logstash 无法从文本文件中读取 json的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我对 ELK 堆栈很陌生，正在尝试使用它.我有一个保存在文本文件中的 json - 只是想将它发送到弹性搜索并使用 kibana 查看.
I am pretty new to ELK stack, trying to play with it. I have a json saved in text file - just want to send it to elastic search and view using kibana.
这是我的配置文件:
input
{
file
{
path => ["C:/Users/vaish/Desktop/sample.txt"]
start_position => beginning
codec => json
}
}
filter {
json {
source => "message"
}
}
output {
stdout { codec => rubydebug }
elasticsearch { hosts => ["localhost:9200"] }
}
这是我在桌面上的 sample.txt 文件.
This is my sample.txt file on desktop.
{"firstname":"bob","lastname":"the builder"}
我在 kibana 中看不到任何输出.
I cannot see any output in kibana.
stdin 的以下简单配置完美运行.
The following simple conf for stdin works perfectly.
input { stdin { } }
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
调试输出:
{:timestamp=>"2015-12-11T12:55:38.257000-0500", :message=>"Reading config file", :config_file=>"C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/bin/first-pipeline.conf", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/agent.rb", :line=>"325", :method=>"local_config"}
{:timestamp=>"2015-12-11T12:55:38.301000-0500", :message=>"Compiled pipeline code:
@inputs = []
@filters = []
@outputs = []
@periodic_flushers = []
@shutdown_flushers = []
@input_file_1 = plugin("input", "file", LogStash::Util.hash_merge_many({ "path" => [("C:/Users/vaish/Desktop/sample.txt")] }, { "start_position" => ("beginning") }, { "codec" => ("json") }, { "sincedb_path" => ("c:/nul") }))
@inputs << @input_file_1
@filter_json_2 = plugin("filter", "json", LogStash::Util.hash_merge_many({ "source" => ("message") }))
@filters << @filter_json_2
@filter_json_2_flush = lambda do |options, &block|
@logger.debug? && @logger.debug("Flushing", :plugin => @filter_json_2)
events = @filter_json_2.flush(options)
return if events.nil? || events.empty?
@logger.debug? && @logger.debug("Flushing", :plugin => @filter_json_2, :events => events)
events.each{|e| block.call(e)}
end
if @filter_json_2.respond_to?(:flush)
@periodic_flushers << @filter_json_2_flush if @filter_json_2.periodic_flush
@shutdown_flushers << @filter_json_2_flush
end
@output_stdout_3 = plugin("output", "stdout", LogStash::Util.hash_merge_many({ "codec" => ("rubydebug") }))
@outputs << @output_stdout_3
@output_elasticsearch_4 = plugin("output", "elasticsearch", LogStash::Util.hash_merge_many({ "hosts" => [("localhost:9200")] }))
@outputs << @output_elasticsearch_4
def filter_func(event)
events = [event]
@logger.debug? && @logger.debug("filter received", :event => event.to_hash)
events = @filter_json_2.multi_filter(events)
events
end
def output_func(event)
@logger.debug? && @logger.debug("output received", :event => event.to_hash)
@output_stdout_3.handle(event)
@output_elasticsearch_4.handle(event)
end", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"38", :method=>"initialize"}
{:timestamp=>"2015-12-11T12:55:38.328000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"input", :name=>"file", :path=>"logstash/inputs/file", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.368000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"json", :path=>"logstash/codecs/json", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.385000-0500", :message=>"config LogStash::Codecs::JSON/@charset = "UTF-8"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.391000-0500", :message=>"config LogStash::Inputs::File/@path = ["C:/Users/vaish/Desktop/sample.txt"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.392000-0500", :message=>"config LogStash::Inputs::File/@start_position = "beginning"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.395000-0500", :message=>"config LogStash::Inputs::File/@codec = <LogStash::Codecs::JSON charset=>"UTF-8">", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.398000-0500", :message=>"config LogStash::Inputs::File/@sincedb_path = "c:/nul"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.400000-0500", :message=>"config LogStash::Inputs::File/@add_field = {}", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.402000-0500", :message=>"config LogStash::Inputs::File/@stat_interval = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.405000-0500", :message=>"config LogStash::Inputs::File/@discover_interval = 15", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.407000-0500", :message=>"config LogStash::Inputs::File/@sincedb_write_interval = 15", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.410000-0500", :message=>"config LogStash::Inputs::File/@delimiter = "\n"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.413000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"filter", :name=>"json", :path=>"logstash/filters/json", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.422000-0500", :message=>"config LogStash::Filters::Json/@source = "message"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.428000-0500", :message=>"config LogStash::Filters::Json/@add_tag = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.435000-0500", :message=>"config LogStash::Filters::Json/@remove_tag = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.438000-0500", :message=>"config LogStash::Filters::Json/@add_field = {}", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.441000-0500", :message=>"config LogStash::Filters::Json/@remove_field = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.443000-0500", :message=>"config LogStash::Filters::Json/@periodic_flush = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.446000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.455000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"rubydebug", :path=>"logstash/codecs/rubydebug", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.466000-0500", :message=>"config LogStash::Codecs::RubyDebug/@metadata = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.523000-0500", :message=>"config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::RubyDebug metadata=>false>", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.524000-0500", :message=>"config LogStash::Outputs::Stdout/@workers = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.527000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"elasticsearch", :path=>"logstash/outputs/elasticsearch", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:39.397000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"plain", :path=>"logstash/codecs/plain", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:39.406000-0500", :message=>"config LogStash::Codecs::Plain/@charset = "UTF-8"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.412000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@hosts = ["localhost:9200"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.414000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@codec = <LogStash::Codecs::Plain charset=>"UTF-8">", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.416000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@workers = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.418000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@index = "logstash-%{+YYYY.MM.dd}"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.419000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@manage_template = true", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.421000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@template_name = "logstash"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.423000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@template_overwrite = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.425000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@parent = nil", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.427000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@flush_size = 500", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.430000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@idle_flush_time = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.432000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@upsert = """, :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.434000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@doc_as_upsert = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.435000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@max_retries = 3", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.437000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_interval = 2", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.439000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_items = 500", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.455000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@action = "index"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.457000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@path = "/"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.459000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@ssl = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.460000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@ssl_certificate_verification = true", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.462000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@sniffing = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.464000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@sniffing_delay = 5", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.760000-0500", :message=>"Registering file input", :path=>["C:/Users/vaish/Desktop/sample.txt"], :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-file-2.0.3/lib/logstash/inputs/file.rb", :line=>"129", :method=>"register"}
{:timestamp=>"2015-12-11T12:55:39.779000-0500", :message=>"_sincedb_open: c:/nul: No such file or directory - c:/nul", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"215", :method=>"_sincedb_open"}
{:timestamp=>"2015-12-11T12:55:39.783000-0500", :message=>"Worker threads expected: 2, worker threads started: 2", :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"186", :method=>"start_filters"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"_globbed_files: C:/Users/vaish/Desktop/sample.txt: glob is: ["C:/Users/vaish/Desktop/sample.txt"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"190", :method=>"_globbed_files"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"_discover_file: C:/Users/vaish/Desktop/sample.txt: new: C:/Users/vaish/Desktop/sample.txt (exclude is [])", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"164", :method=>"_discover_file"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"Normalizing http path", :path=>"/", :normalized=>"/", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb", :line=>"14", :method=>"build"}
{:timestamp=>"2015-12-11T12:55:39.819000-0500", :message=>"Elasticsearch HTTP client options", :hosts=>["http://localhost:9200/"], :ssl=>nil, :transport_options=>{:socket_timeout=>0, :request_timeout=>0, :proxy=>nil}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::Manticore, :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client.rb", :line=>"147", :method=>"build_client"}
{:timestamp=>"2015-12-11T12:55:39.819000-0500", :message=>"_open_file: C:/Users/vaish/Desktop/sample.txt: opening", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"118", :method=>"_open_file"}
{:timestamp=>"2015-12-11T12:55:39.840000-0500", :message=>"Using mapping template from", :path=>nil, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb", :line=>"6", :method=>"install_template"}
{:timestamp=>"2015-12-11T12:55:39.847000-0500", :message=>"C:/Users/vaish/Desktop/sample.txt: initial create, no sincedb, seeking to beginning of file", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"155", :method=>"_open_file"}
{:timestamp=>"2015-12-11T12:55:39.852000-0500", :message=>"writing sincedb (delta since last write = 1449856539)", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"195", :method=>"_read_file"}
{:timestamp=>"2015-12-11T12:55:39.855000-0500", :message=>"C:/Users/vaish/Desktop/sample.txt: file grew, old size 0, new size 44", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"121", :method=>"each"}
{:timestamp=>"2015-12-11T12:55:40.103000-0500", :message=>"Attempting to install template", :manage_template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "doc_values"=>true, "ignore_above"=>256}}}}}, {"float_fields"=>{"match"=>"*", "match_mapping_type"=>"float", "mapping"=>{"type"=>"float", "doc_values"=>true}}}, {"double_fields"=>{"match"=>"*", "match_mapping_type"=>"double", "mapping"=>{"type"=>"double", "doc_values"=>true}}}, {"byte_fields"=>{"match"=>"*", "match_mapping_type"=>"byte", "mapping"=>{"type"=>"byte", "doc_values"=>true}}}, {"short_fields"=>{"match"=>"*", "match_mapping_type"=>"short", "mapping"=>{"type"=>"short", "doc_values"=>true}}}, {"integer_fields"=>{"match"=>"*", "match_mapping_type"=>"integer", "mapping"=>{"type"=>"integer", "doc_values"=>true}}}, {"long_fields"=>{"match"=>"*", "match_mapping_type"=>"long", "mapping"=>{"type"=>"long", "doc_values"=>true}}}, {"date_fields"=>{"match"=>"*", "match_mapping_type"=>"date", "mapping"=>{"type"=>"date", "doc_values"=>true}}}, {"geo_point_fields"=>{"match"=>"*", "match_mapping_type"=>"geo_point", "mapping"=>{"type"=>"geo_point", "doc_values"=>true}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "doc_values"=>true}, "@version"=>{"type"=>"string", "index"=>"not_analyzed", "doc_values"=>true}, "geoip"=>{"type"=>"object", "dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip", "doc_values"=>true}, "location"=>{"type"=>"geo_point", "doc_values"=>true}, "latitude"=>{"type"=>"float", "doc_values"=>true}, "longitude"=>{"type"=>"float", "doc_values"=>true}}}}}}}, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb", :line=>"8", :method=>"install_template"}
{:timestamp=>"2015-12-11T12:55:40.272000-0500", :message=>"Found existing Elasticsearch template. Skipping template management", :name=>"logstash", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client.rb", :line=>"30", :method=>"template_install"}
{:timestamp=>"2015-12-11T12:55:40.274000-0500", :message=>"New Elasticsearch output", :class=>"LogStash::Outputs::ElasticSearch", :hosts=>["localhost:9200"], :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb", :line=>"19", :method=>"register"}
{:timestamp=>"2015-12-11T12:55:40.276000-0500", :message=>"Pipeline started", :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"109", :method=>"run"}
{:timestamp=>"2015-12-11T12:55:41.297000-0500", :message=>"Flushing buffer at interval", :instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x456f5d26 @operations_mutex=#<Mutex:0x4ba8abfb>, @max_size=500, @operations_lock=#<Java::JavaUtilConcurrentLocks::ReentrantLock:0x4f973656>, @submit_proc=#<Proc:0x352c5bd9@C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb:55>, @logger=#<Cabin::Channel:0x3b55c9f4 @metrics=#<Cabin::Metrics:0x4d066d4e @metrics_lock=#<Mutex:0x6aba5e8f>, @metrics={}, @channel=#<Cabin::Channel:0x3b55c9f4 ...>>, @subscriber_lock=#<Mutex:0x3d9f3292>, @level=:debug, @subscribers={12592=>#<Cabin::Outputs::IO:0x560222ed @io=#<IO:fd 1>, @lock=#<Mutex:0x6ef44084>>}, @data={}>, @last_flush=2015-12-11 12:55:40 -0500, @flush_interval=1, @stopping=#<Concurrent::AtomicBoolean:0x2abbf46a>, @buffer=[], @flush_thread=#<Thread:0xb4f237 run>>", :interval=>1, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}
{:timestamp=>"2015-12-11T12:55:42.331000-0500", :message=>"Flushing buffer at interval", :instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x456f5d26 @operations_mutex=#<Mutex:0x4ba8abfb>, @max_size=500, @operations_lock=#<Java::JavaUtilConcurrentLocks::ReentrantLock:0x4f973656>, @submit_proc=#<Proc:0x352c5bd9@C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb:55>, @logger=#<Cabin::Channel:0x3b55c9f4 @metrics=#<Cabin::Metrics:0x4d066d4e @metrics_lock=#<Mutex:0x6aba5e8f>, @metrics={}, @channel=#<Cabin::Channel:0x3b55c9f4 ...>>, @subscriber_lock=#<Mutex:0x3d9f3292>, @level=:debug, @subscribers={12592=>#<Cabin::Outputs::IO:0x560222ed @io=#<IO:fd 1>, @lock=#<Mutex:0x6ef44084>>}, @data={}>, @last_flush=2015-12-11 12:55:41 -0500, @flush_interval=1, @stopping=#<Concurrent::AtomicBoolean:0x2abbf46a>, @buffer=[], @flush_thread=#<Thread:0xb4f237 run>>", :interval=>1, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}
推荐答案
你需要修改你的file 输入过滤器 通过设置 sincedb_path property to null.Otherwise, Logstash will only start reading the file where it left off the last time you ran it.
You need to modify your file input filter to not keep track of where the file has been read so far by setting the sincedb_path property to null. Otherwise, Logstash will only start reading the file where it left off the last time you ran it.
input {
file {
path => ["C:/Users/vaish/Desktop/sample.txt"]
start_position => beginning
codec => json
sincedb_path => "/dev/null" <--- add this line
}
}
更新
Another solution would be to simply use the stdin input
Another solution would be to simply use the stdin input
input {
stdin {}
}
and input redirection on the command line.
and input redirection on the command line.
logstash.bat -c first-pipeline.conf < C:/Users/vaish/Desktop/sample.txt
这篇关于Logstash 无法从文本文件中读取 json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
