StormCrawler 无法连接到 ElasticSearch [英] StormCrawler cannot connect to ElasticSearch
本文介绍了StormCrawler 无法连接到 ElasticSearch的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
运行命令时:
storm jar target/crawlIndexer-1.0-SNAPSHOT.jar org.apache.storm.flux.Flux --local es-injector.flux --sleep 86400000
我收到一条错误消息:
8710 [Thread-26-status-executor[4 4]] ERROR c.d.s.e.p.StatusUpdaterBolt -
Can't connect to ElasticSearch
在浏览器中运行 http://localhost:9200/ 时,ES 成功加载.Kibana 还连接到 ES.所以它一定只是从 StromCrawler 到 ElasticSearch 的连接.
When running http://localhost:9200/ in browser ES successfully loads up. Kibana also connects to ES. So it must just be the connection from StromCrawler to ElasticSearch.
可能是什么问题?
完整错误的片段:
8710 [Thread-26-status-executor[4 4]] ERROR c.d.s.e.p.StatusUpdaterBolt - Can't connect to ElasticSearch
java.lang.RuntimeException: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:107) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:151) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:141) [crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:482) [storm-core-1.1.0.jar:1.1.0]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_151]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[?:1.8.0_151]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_151]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_151]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:104) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
... 6 more
8710 [Thread-18-__metricscom.digitalpebble.stormcrawler.elasticsearch.metrics.MetricsConsumer-executor[2 2]] ERROR c.d.s.e.m.MetricsConsumer - Can't connect to ElasticSearch
java.lang.RuntimeException: java.net.UnknownHostException: http
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:107) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:151) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:133) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.metrics.MetricsConsumer.prepare(MetricsConsumer.java:68) [crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.metric.MetricsConsumerBolt.prepare(MetricsConsumerBolt.java:77) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:482) [storm-core-1.1.0.jar:1.1.0]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.net.UnknownHostException: http
at java.net.InetAddress.getAllByName0(InetAddress.java:1280) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_151]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_151]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:104) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
... 8 more
8714 [Thread-26-status-executor[4 4]] ERROR o.a.s.util - Async loop died!
java.lang.RuntimeException: java.lang.RuntimeException: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at com.digitalpebble.stormcrawler.elasticsearch.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:145) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) ~[storm-core-1.1.0.jar:1.1.0]
也许我需要在 elasticsearch.yml 或 es-conf.yml 中修改一些内容?(它们都有默认设置)
Perhaps there is something I need to modify in elasticsearch.yml or es-conf.yml? (They both have default settings)
推荐答案
通过玩弄(玩了好几个小时)让应用程序开始工作.
Got the application to work by playing around (for many hours).
步骤:
- 在 maven 文件中将 Storm-Crawler-Core 和 Storm-Crawler-Elastic-Search 恢复到 1.5 版.
- 使用来自 Storm-Crawler-Elastic-Search 的 1.5 通量/yaml 配置文件版本
- 使用 Elastic Search 5.4.0
- 使用 Kibana 5.4.0
这篇关于StormCrawler 无法连接到 ElasticSearch的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文