弹性搜索聚合在升级到1.7.3之后被破坏 [英] Elasticsearch Aggregation Broken after upgrade to 1.7.3

问题描述

在升级到1.7.3之前，它正在工作，但现在告诉我我的性别数据太大，我运行了

It was working before the upgrade to 1.7.3 but now it is telling me my "Data too large for [Gender]. I ran the

curl -XGET localhost:9200/_nodes/stats/indices/fielddata?fields=*

它生成

{
    {"fielddata":{"memory_size_in_bytes":642066528,"evictions":0,
    "fields":{"Markers":{"memory_size_in_bytes":196538816},
     "RegistrationDate":{"memory_size_in_bytes":101759288},
     "Abbreviation":{"memory_size_in_bytes":185815224},
     "Gender":{"memory_size_in_bytes":52988320},
     "Birthdate":{"memory_size_in_bytes":104956384},
     "buildNum":{"memory_size_in_bytes":8496}
     }
}

性别只是一个字符，所以如何可以增长那么大吗？

Gender is only a single character so how could it grow that large?

阅读 https://www.elastic.co/guide /en/elasticsearch/guide/current/_limiting_memory_usage.html 似乎表明它不应该那么大。它的值只有M，F，U。任何想法的原因？

Reading https://www.elastic.co/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html seems to indicate that it should not be that big. It only has M, F, U for values. Any thoughts to the reason?

标记

nested:  UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [Gender] would be larger than limit of [633785548/604.4mb]];

推荐答案

您可能会遇到 CircuitBreakingException 。其主要原因是您的堆几乎已满，并且没有足够的可用内存来提供请求。这不一定意味着 Gender 的值占用了整个内存，只是ES无法加载所需的性别的fielddata 字段与剩余的可用内存。

You might be encountering a CircuitBreakingException. The main reason for this is that your heap is almost full and there was not enough memory available to serve the request. It doesn't necessarily mean that the Gender values take up the whole memory, just that ES was unable to load all the fielddata it needed for the Gender field with what remains of the available memory.

默认情况下，没有字段数据驱逐发生（提示驱逐： 0 在您的输出）如果没有为fielddata缓存设置特定的限制，默认情况下没有设置限制。因此，您应该尝试将 indices.fielddata.cache.size 设置为相对值（例如10％）或绝对值（例如3GB）。

By default, no fielddata evictions happen (hint "evictions":0 in your output) if there's no specific limit set for the fielddata cache and by default no limit is set. So you should try setting the indices.fielddata.cache.size to a relative (e.g. 10%) or an absolute value (e.g. 3GB).

您可以在 elasticsearch.yml 文件（每个节点上）设置它，并重新启动节点，或者使用

You can either set it in the elasticsearch.yml file (on each node) and restart your nodes or do it dynamically with

curl -XPUT localhost:9200/_cluster/settings -d '{
    "persistent" : {
        "indices.fielddata.cache.size" : "20%"
    }
}'

请注意，设置太低的值将对您的群集产生不利影响，因为将会有大量的驱逐，并且fielddata缓存将不得不经常重建。所以你可能需要尝试一下，直到找到正确的值。

Note that setting a value that is too low will negatively impact your cluster since there will be a lot of evictions and fielddata cache will have to be rebuilt too often. So you probably need to experiment a bit until you find the right value.

这篇关于弹性搜索聚合在升级到1.7.3之后被破坏的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！