如何将ElasticSearch查询传递给hadoop [英] How to pass ElasticSearch query to hadoop

问题描述

我正在开展一个项目，以便在Hadoop上进行搜索。它的工作原理是首先将查询传递给ES，然后将结果ID传回SQL，如下所示：

I am working on a project to allow searching on Hadoop. How it works is the query is first passed to ES, then the resulting IDs are passed back to SQL, like so:

SELECT ... WHERE id in [1,24,383,2912,...]

我们有100或1000个ids返回到SQL-on-Hadoop，但是如果我们有100K或1M + ids来传回，则不会那么好。将结果集（ids？的列表）从ES返回到SQL是一些替代方法？

This works well when we have 100 or 1000 ids to pass back to SQL-on-Hadoop, but not so well if we have 100K or 1M+ ids to pass back. What are some alternative approaches to pass the result set (list of ids?) from ES back to SQL?

推荐答案

插入所有弹性搜索 ids到一个单独的表，然后加入源表。

Insert all elastic search ids in to a separate table then join it with source table.

例如

CREATE TABLE ELASTIC_SEARCH_ID(id INT);

SELECT ... FROM SOURCE S JOIN ELASTIC_SEARCH_ID ES ON S.ID=ES.ID;

这篇关于如何将ElasticSearch查询传递给hadoop的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！