如何在一台服务器上配置弹性搜索集群以获得最佳搜索效果 [英] How to config elasticsearch cluster on one server to get the best performace on search

问题描述

我刚刚弹性搜索。我有一个弹性搜索索引约30万项目。对于另一个表中的6000万条记录中的每一个，我需要对此ES索引进行复杂的查询。

现在，这非常慢（使1000个查询花费200秒）。我需要关于如何配置我的弹性搜索集群来处理大量查询的建议。

我的服务器：

  8核心
 8GB RAM 
 SSD硬件
  

我想配置弹性搜索来处理来自ruby的1000个并发搜索请求。 （我想并行搜索1000个项目）

我尝试使用默认配置

我认为默认情况下，elasticesearch只能处理大约10-20个并发搜索请求。它使用小cpu和ram。因此，我想我可以改进它。

我只能运行100个线程从红宝石搜索1000个项目，它需要大约200秒。如果我从ruby增加到1000个线程，ES返回超时错误消息。

我运行一个主节点

  ES_HEAP_SIZE = 2G 
 
 indices.fielddata.cache.size：1g 
 
线程池：
搜索：
类型：固定
大小：200 
 queue_size：400 
 
 share：5 
 
副本：1 
  

从红宝石运行100个线程搜索1000个项目仍然需要200s。

我添加3个新节点作为数据节点这个服务器。

从红宝石运行100个线程搜索1000个项目仍需要200s或更长时间。

我的谷歌和阅读从一些帖子。人们说，创建更多的分片将使搜索变得缓慢。

如何改进我的搜索查询？

非常感谢！

解决方案

您将要观看此视频：

http://www.elasticsearch.org/webinars/elasticsearch-飞行前检查清单/

ES的默认值非常适合开发但不适合生产。您真正需要做的一件事是为服务器上的可用内存提供50％的JVM。这个视频有很多其他很棒的提示。

I am new to elasticsearch. I have an Elasticsearch index of about 300,000 items. For each of the 60 million records in another table, I need to make a complex query to this ES index.

Right now, it is extremely slow (making 1000 queries would take 200 seconds). I need advice on how to configure my elasticsearch cluster to handle a large volume of queries.

My server:

8 core
8GB ram
SSD Hardware

I want to config elasticsearch to handle 1000 concurrent search requests from ruby. (I want to search 1000 items in parallel).

I have try with the default config

I think by default, elasticesearch can only handle about 10-20 concurrent search request. It use little cpu and ram. Therefore, I think I could improve it.

I could only run 100 threads from ruby to search 1000 items and it takes about 200 seconds. If I increase to 1000 threads from ruby, ES returns timeout error message.

I run a master node with

ES_HEAP_SIZE=2G

indices.fielddata.cache.size: 1g 

threadpool:   
   search:
      type: fixed
      size: 200
      queue_size: 400

shares: 5 

replicas: 1

Running 100 threads from ruby to search 1000 items still takes 200s.

I add 3 new nodes as data nodes on this server.

Running 100 threads from ruby to search 1000 items still takes 200s or more.

I google and read from some posts. People say that create more shards will make search become slow.

How can I improve my search query?

Many thanks!

解决方案

you're going to want to watch this video:

http://www.elasticsearch.org/webinars/elasticsearch-pre-flight-checklist/

The defaults for ES are great for development but not production. The one thing that you really need to do is give the JVM 50% of the available memory on the server. That video has lots of other great tips.

这篇关于如何在一台服务器上配置弹性搜索集群以获得最佳搜索效果的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！