Logstash无法从文本文件读取json [英] Logstash unable to read json from text file
本文介绍了Logstash无法从文本文件读取json的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
我对ELK堆栈还很陌生，尝试使用它.我有一个保存在文本文件中的json-只想将其发送到弹性搜索并使用kibana进行查看.
I am pretty new to ELK stack, trying to play with it. I have a json saved in text file - just want to send it to elastic search and view using kibana.
这是我的conf文件:
This is my conf file :
input
{
file
{
path => ["C:/Users/vaish/Desktop/sample.txt"]
start_position => beginning
codec => json
}
}
filter {
json {
source => "message"
}
}
output {
stdout { codec => rubydebug }
elasticsearch { hosts => ["localhost:9200"] }
}
这是我在桌面上的sample.txt文件.
This is my sample.txt file on desktop.
{"firstname":"bob","lastname":"the builder"}
在kibana中看不到任何输出.
I cannot see any output in kibana.
以下用于stdin的简单conf可以很好地工作.
The following simple conf for stdin works perfectly.
input { stdin { } }
output {
elasticsearch { hosts => ["localhost:9200"] }
stdout { codec => rubydebug }
}
调试输出:
{:timestamp=>"2015-12-11T12:55:38.257000-0500", :message=>"Reading config file", :config_file=>"C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/bin/first-pipeline.conf", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/agent.rb", :line=>"325", :method=>"local_config"}
{:timestamp=>"2015-12-11T12:55:38.301000-0500", :message=>"Compiled pipeline code:\n @inputs = []\n @filters = []\n @outputs = []\n @periodic_flushers = []\n @shutdown_flushers = []\n\n @input_file_1 = plugin(\"input\", \"file\", LogStash::Util.hash_merge_many({ \"path\" => [(\"C:/Users/vaish/Desktop/sample.txt\")] }, { \"start_position\" => (\"beginning\") }, { \"codec\" => (\"json\") }, { \"sincedb_path\" => (\"c:/nul\") }))\n\n @inputs << @input_file_1\n\n @filter_json_2 = plugin(\"filter\", \"json\", LogStash::Util.hash_merge_many({ \"source\" => (\"message\") }))\n\n @filters << @filter_json_2\n\n @filter_json_2_flush = lambda do |options, &block|\n @logger.debug? && @logger.debug(\"Flushing\", :plugin => @filter_json_2)\n\n events = @filter_json_2.flush(options)\n\n return if events.nil? || events.empty?\n\n @logger.debug? && @logger.debug(\"Flushing\", :plugin => @filter_json_2, :events => events)\n\n \n\n events.each{|e| block.call(e)}\n end\n\n if @filter_json_2.respond_to?(:flush)\n @periodic_flushers << @filter_json_2_flush if @filter_json_2.periodic_flush\n @shutdown_flushers << @filter_json_2_flush\n end\n\n @output_stdout_3 = plugin(\"output\", \"stdout\", LogStash::Util.hash_merge_many({ \"codec\" => (\"rubydebug\") }))\n\n @outputs << @output_stdout_3\n\n @output_elasticsearch_4 = plugin(\"output\", \"elasticsearch\", LogStash::Util.hash_merge_many({ \"hosts\" => [(\"localhost:9200\")] }))\n\n @outputs << @output_elasticsearch_4\n\n def filter_func(event)\n events = [event]\n @logger.debug? && @logger.debug(\"filter received\", :event => event.to_hash)\n events = @filter_json_2.multi_filter(events)\n \n events\n end\n def output_func(event)\n @logger.debug? && @logger.debug(\"output received\", :event => event.to_hash)\n @output_stdout_3.handle(event)\n @output_elasticsearch_4.handle(event)\n \n end", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"38", :method=>"initialize"}
{:timestamp=>"2015-12-11T12:55:38.328000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"input", :name=>"file", :path=>"logstash/inputs/file", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.368000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"json", :path=>"logstash/codecs/json", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.385000-0500", :message=>"config LogStash::Codecs::JSON/@charset = \"UTF-8\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.391000-0500", :message=>"config LogStash::Inputs::File/@path = [\"C:/Users/vaish/Desktop/sample.txt\"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.392000-0500", :message=>"config LogStash::Inputs::File/@start_position = \"beginning\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.395000-0500", :message=>"config LogStash::Inputs::File/@codec = <LogStash::Codecs::JSON charset=>\"UTF-8\">", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.398000-0500", :message=>"config LogStash::Inputs::File/@sincedb_path = \"c:/nul\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.400000-0500", :message=>"config LogStash::Inputs::File/@add_field = {}", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.402000-0500", :message=>"config LogStash::Inputs::File/@stat_interval = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.405000-0500", :message=>"config LogStash::Inputs::File/@discover_interval = 15", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.407000-0500", :message=>"config LogStash::Inputs::File/@sincedb_write_interval = 15", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.410000-0500", :message=>"config LogStash::Inputs::File/@delimiter = \"\\n\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.413000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"filter", :name=>"json", :path=>"logstash/filters/json", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.422000-0500", :message=>"config LogStash::Filters::Json/@source = \"message\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.428000-0500", :message=>"config LogStash::Filters::Json/@add_tag = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.435000-0500", :message=>"config LogStash::Filters::Json/@remove_tag = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.438000-0500", :message=>"config LogStash::Filters::Json/@add_field = {}", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.441000-0500", :message=>"config LogStash::Filters::Json/@remove_field = []", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.443000-0500", :message=>"config LogStash::Filters::Json/@periodic_flush = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.446000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"stdout", :path=>"logstash/outputs/stdout", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.455000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"rubydebug", :path=>"logstash/codecs/rubydebug", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:38.466000-0500", :message=>"config LogStash::Codecs::RubyDebug/@metadata = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.523000-0500", :message=>"config LogStash::Outputs::Stdout/@codec = <LogStash::Codecs::RubyDebug metadata=>false>", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.524000-0500", :message=>"config LogStash::Outputs::Stdout/@workers = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:38.527000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"output", :name=>"elasticsearch", :path=>"logstash/outputs/elasticsearch", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:39.397000-0500", :message=>"Plugin not defined in namespace, checking for plugin file", :type=>"codec", :name=>"plain", :path=>"logstash/codecs/plain", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/plugin.rb", :line=>"76", :method=>"lookup"}
{:timestamp=>"2015-12-11T12:55:39.406000-0500", :message=>"config LogStash::Codecs::Plain/@charset = \"UTF-8\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.412000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@hosts = [\"localhost:9200\"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.414000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@codec = <LogStash::Codecs::Plain charset=>\"UTF-8\">", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.416000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@workers = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.418000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@index = \"logstash-%{+YYYY.MM.dd}\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.419000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@manage_template = true", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.421000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@template_name = \"logstash\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.423000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@template_overwrite = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.425000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@parent = nil", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.427000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@flush_size = 500", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.430000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@idle_flush_time = 1", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.432000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@upsert = \"\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.434000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@doc_as_upsert = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.435000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@max_retries = 3", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.437000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_interval = 2", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.439000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@retry_max_items = 500", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.455000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@action = \"index\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.457000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@path = \"/\"", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.459000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@ssl = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.460000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@ssl_certificate_verification = true", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.462000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@sniffing = false", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.464000-0500", :message=>"config LogStash::Outputs::ElasticSearch/@sniffing_delay = 5", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/config/mixin.rb", :line=>"122", :method=>"config_init"}
{:timestamp=>"2015-12-11T12:55:39.760000-0500", :message=>"Registering file input", :path=>["C:/Users/vaish/Desktop/sample.txt"], :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-input-file-2.0.3/lib/logstash/inputs/file.rb", :line=>"129", :method=>"register"}
{:timestamp=>"2015-12-11T12:55:39.779000-0500", :message=>"_sincedb_open: c:/nul: No such file or directory - c:/nul", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"215", :method=>"_sincedb_open"}
{:timestamp=>"2015-12-11T12:55:39.783000-0500", :message=>"Worker threads expected: 2, worker threads started: 2", :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"186", :method=>"start_filters"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"_globbed_files: C:/Users/vaish/Desktop/sample.txt: glob is: [\"C:/Users/vaish/Desktop/sample.txt\"]", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"190", :method=>"_globbed_files"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"_discover_file: C:/Users/vaish/Desktop/sample.txt: new: C:/Users/vaish/Desktop/sample.txt (exclude is [])", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"164", :method=>"_discover_file"}
{:timestamp=>"2015-12-11T12:55:39.809000-0500", :message=>"Normalizing http path", :path=>"/", :normalized=>"/", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client_builder.rb", :line=>"14", :method=>"build"}
{:timestamp=>"2015-12-11T12:55:39.819000-0500", :message=>"Elasticsearch HTTP client options", :hosts=>["http://localhost:9200/"], :ssl=>nil, :transport_options=>{:socket_timeout=>0, :request_timeout=>0, :proxy=>nil}, :transport_class=>Elasticsearch::Transport::Transport::HTTP::Manticore, :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client.rb", :line=>"147", :method=>"build_client"}
{:timestamp=>"2015-12-11T12:55:39.819000-0500", :message=>"_open_file: C:/Users/vaish/Desktop/sample.txt: opening", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"118", :method=>"_open_file"}
{:timestamp=>"2015-12-11T12:55:39.840000-0500", :message=>"Using mapping template from", :path=>nil, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb", :line=>"6", :method=>"install_template"}
{:timestamp=>"2015-12-11T12:55:39.847000-0500", :message=>"C:/Users/vaish/Desktop/sample.txt: initial create, no sincedb, seeking to beginning of file", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"155", :method=>"_open_file"}
{:timestamp=>"2015-12-11T12:55:39.852000-0500", :message=>"writing sincedb (delta since last write = 1449856539)", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/tail.rb", :line=>"195", :method=>"_read_file"}
{:timestamp=>"2015-12-11T12:55:39.855000-0500", :message=>"C:/Users/vaish/Desktop/sample.txt: file grew, old size 0, new size 44", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/filewatch-0.6.7/lib/filewatch/watch.rb", :line=>"121", :method=>"each"}
{:timestamp=>"2015-12-11T12:55:40.103000-0500", :message=>"Attempting to install template", :manage_template=>{"template"=>"logstash-*", "settings"=>{"index.refresh_interval"=>"5s"}, "mappings"=>{"_default_"=>{"_all"=>{"enabled"=>true, "omit_norms"=>true}, "dynamic_templates"=>[{"message_field"=>{"match"=>"message", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}}}}, {"string_fields"=>{"match"=>"*", "match_mapping_type"=>"string", "mapping"=>{"type"=>"string", "index"=>"analyzed", "omit_norms"=>true, "fielddata"=>{"format"=>"disabled"}, "fields"=>{"raw"=>{"type"=>"string", "index"=>"not_analyzed", "doc_values"=>true, "ignore_above"=>256}}}}}, {"float_fields"=>{"match"=>"*", "match_mapping_type"=>"float", "mapping"=>{"type"=>"float", "doc_values"=>true}}}, {"double_fields"=>{"match"=>"*", "match_mapping_type"=>"double", "mapping"=>{"type"=>"double", "doc_values"=>true}}}, {"byte_fields"=>{"match"=>"*", "match_mapping_type"=>"byte", "mapping"=>{"type"=>"byte", "doc_values"=>true}}}, {"short_fields"=>{"match"=>"*", "match_mapping_type"=>"short", "mapping"=>{"type"=>"short", "doc_values"=>true}}}, {"integer_fields"=>{"match"=>"*", "match_mapping_type"=>"integer", "mapping"=>{"type"=>"integer", "doc_values"=>true}}}, {"long_fields"=>{"match"=>"*", "match_mapping_type"=>"long", "mapping"=>{"type"=>"long", "doc_values"=>true}}}, {"date_fields"=>{"match"=>"*", "match_mapping_type"=>"date", "mapping"=>{"type"=>"date", "doc_values"=>true}}}, {"geo_point_fields"=>{"match"=>"*", "match_mapping_type"=>"geo_point", "mapping"=>{"type"=>"geo_point", "doc_values"=>true}}}], "properties"=>{"@timestamp"=>{"type"=>"date", "doc_values"=>true}, "@version"=>{"type"=>"string", "index"=>"not_analyzed", "doc_values"=>true}, "geoip"=>{"type"=>"object", "dynamic"=>true, "properties"=>{"ip"=>{"type"=>"ip", "doc_values"=>true}, "location"=>{"type"=>"geo_point", "doc_values"=>true}, "latitude"=>{"type"=>"float", "doc_values"=>true}, "longitude"=>{"type"=>"float", "doc_values"=>true}}}}}}}, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/template_manager.rb", :line=>"8", :method=>"install_template"}
{:timestamp=>"2015-12-11T12:55:40.272000-0500", :message=>"Found existing Elasticsearch template. Skipping template management", :name=>"logstash", :level=>:debug, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/http_client.rb", :line=>"30", :method=>"template_install"}
{:timestamp=>"2015-12-11T12:55:40.274000-0500", :message=>"New Elasticsearch output", :class=>"LogStash::Outputs::ElasticSearch", :hosts=>["localhost:9200"], :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb", :line=>"19", :method=>"register"}
{:timestamp=>"2015-12-11T12:55:40.276000-0500", :message=>"Pipeline started", :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb", :line=>"109", :method=>"run"}
{:timestamp=>"2015-12-11T12:55:41.297000-0500", :message=>"Flushing buffer at interval", :instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x456f5d26 @operations_mutex=#<Mutex:0x4ba8abfb>, @max_size=500, @operations_lock=#<Java::JavaUtilConcurrentLocks::ReentrantLock:0x4f973656>, @submit_proc=#<Proc:0x352c5bd9@C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb:55>, @logger=#<Cabin::Channel:0x3b55c9f4 @metrics=#<Cabin::Metrics:0x4d066d4e @metrics_lock=#<Mutex:0x6aba5e8f>, @metrics={}, @channel=#<Cabin::Channel:0x3b55c9f4 ...>>, @subscriber_lock=#<Mutex:0x3d9f3292>, @level=:debug, @subscribers={12592=>#<Cabin::Outputs::IO:0x560222ed @io=#<IO:fd 1>, @lock=#<Mutex:0x6ef44084>>}, @data={}>, @last_flush=2015-12-11 12:55:40 -0500, @flush_interval=1, @stopping=#<Concurrent::AtomicBoolean:0x2abbf46a>, @buffer=[], @flush_thread=#<Thread:0xb4f237 run>>", :interval=>1, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}
{:timestamp=>"2015-12-11T12:55:42.331000-0500", :message=>"Flushing buffer at interval", :instance=>"#<LogStash::Outputs::ElasticSearch::Buffer:0x456f5d26 @operations_mutex=#<Mutex:0x4ba8abfb>, @max_size=500, @operations_lock=#<Java::JavaUtilConcurrentLocks::ReentrantLock:0x4f973656>, @submit_proc=#<Proc:0x352c5bd9@C:/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/common.rb:55>, @logger=#<Cabin::Channel:0x3b55c9f4 @metrics=#<Cabin::Metrics:0x4d066d4e @metrics_lock=#<Mutex:0x6aba5e8f>, @metrics={}, @channel=#<Cabin::Channel:0x3b55c9f4 ...>>, @subscriber_lock=#<Mutex:0x3d9f3292>, @level=:debug, @subscribers={12592=>#<Cabin::Outputs::IO:0x560222ed @io=#<IO:fd 1>, @lock=#<Mutex:0x6ef44084>>}, @data={}>, @last_flush=2015-12-11 12:55:41 -0500, @flush_interval=1, @stopping=#<Concurrent::AtomicBoolean:0x2abbf46a>, @buffer=[], @flush_thread=#<Thread:0xb4f237 run>>", :interval=>1, :level=>:info, :file=>"/Users/vaish/Desktop/OCIFitbit/logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-2.2.0-java/lib/logstash/outputs/elasticsearch/buffer.rb", :line=>"90", :method=>"interval_flush"}
推荐答案
You need to modify your file input filter to not keep track of where the file has been read so far by setting the sincedb_path property to null. Otherwise, Logstash will only start reading the file where it left off the last time you ran it.
input {
file {
path => ["C:/Users/vaish/Desktop/sample.txt"]
start_position => beginning
codec => json
sincedb_path => "/dev/null" <--- add this line
}
}
更新
另一种解决方案是仅使用stdin输入
Another solution would be to simply use the stdin input
input {
stdin {}
}
，然后在命令行上输入重定向.
and input redirection on the command line.
logstash.bat -c first-pipeline.conf < C:/Users/vaish/Desktop/sample.txt
这篇关于Logstash无法从文本文件读取json的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
