Elasticsearch/Storm 集成方法 [英] Elasticsearch / Storm integration methods

问题描述

寻找 Elasticsearch 和 Apache Storm 之间的简单集成路径.对此的支持包含在 elasticsearch-hadoop 库中，但这给 Hadoop 堆栈带来了大量依赖:从 Hive 到 Cascading，我根本不需要这些.有没有人在没有引入 elasticsearch-hadoop 的情况下成功地进行了这种集成?谢谢.

Looking for a simple integration path between Elasticsearch and Apache Storm. Support for this is included in the elasticsearch-hadoop library, but this brings tons of dependencies on the Hadoop stack: from Hive to Cascading, that I simply don't need. Has anyone out there succeeded in this integration without bringing in elasticsearch-hadoop? Thanks.

推荐答案

在我的项目中，我们使用 rabbitmq river 用于索引风暴输出.这是写入 elasticsearch 的非常有效和方便的方式.您基本上将消息放入队列，其余的由河流完成.如果有什么东西卡住了，数据就会简单地缓存在队列中.

In my project we're using rabbitmq river for indexing the storm output. It's very efficient and convenient way to write to elasticsearch. You basically put the messages to the queue and the river does the rest. If something gets stucked the data are simply buffered on the queue.

所以我想说，使用这种河流方法来编写和使用 elasticsearch Java API 来读取，就像 Kit Menke 建议的(或 Jest 客户端，我们发现这很酷，它提供基于 ApacheHttpAsyncClient 的异步 API，尽管我们不是在 Storm 拓扑中而是在不同的服务中从 elasticsearch 中读取数据).

So I would say, use this river approach for writing and elasticsearch Java API for reading, like Kit Menke suggests (or the Jest client, we've found this cool and it offers async API basing on ApacheHttpAsyncClient, though we're not reading from elasticsearch in storm topology but in different services).

这篇关于Elasticsearch/Storm 集成方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！