尝试在Elasticsearch中设置max_gram和min_gram [英] Trying to set the max_gram and min_gram in Elasticsearch
问题描述
我正在尝试在Ubuntu 16.04 EC2服务器上部署Ruby on Rails应用程序，但是给出了关于Elasticsearch上max_gram和min_gram之间差异的错误，我没有任何Elasticsearch经验，所以我在这里完全迷失了，我需要一些指导来做到这一点，并学习如何设置它以避免将来出现此问题.
我第一次进行部署时，出现错误，拒绝与localhost:9200的连接，因此我不得不检查服务是否正在运行，甚至检查防火墙，最后我必须进行全新安装并配置所有内容在elasticsearch.yml上，现在可以运行并运行，但是当我再次尝试部署时出现错误，在Internet上进行了大量搜索，虽然有很多文档，但是我仍然无法在哪里设置这些值./p>
这是即时通讯上出现的错误:
----->正在迁移数据库...耙子流产了！StandardError:发生错误，所有以后的迁移都被取消:[400] {错误":{"root_cause":[{类型":"illegal_argument_exception"，原因":"NGram Tokenizer中的max_gram和min_gram之间的差必须小于或等于:[1]，但[49].可以通过更改[index.max_ngram_diff]索引级别设置来设置此限制.}]，" type:" illegal_argument_exception，" reason:" NGram Tokenizer中的max_gram和min_gram之间的差异必须较小等于或等于:[1]但为[49].可以通过更改[index.max_ngram_diff]索引级别设置来设置此限制.}，"状态:400}/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/base.rb:205:in`__raise_transport_error'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/base.rb:323:in'perform_request'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/http/faraday.rb:20:in'perform_request'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/client.rb:131:in'perform_request'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api/namespace/common.rb:21:in'perform_request'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api/actions/indices/create.rb:86:in'create'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:16:在'create'中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:203:位于"create_index"中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:270:在"reindex_scope"中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:196:在`reindex'中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/model.rb:59:在`searchkick_reindex'中/home/deploy/catalogindustry/releases/20190807135404/db/migrate/20180405153226_validated_true.rb:4:in`change'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:789:在`exec_migration'中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:773:位于迁移中的块(2个级别)"中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:772:在在迁移中阻止"/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:398:在withwithconnection中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:771:in`migrate'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:951:在'migrate'中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1232:in`execute_migration_in_transaction中的块"/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1302:in`ddl_transaction'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1231:in`execute_migration_in_transaction'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1203:inmigrate_without_lock中的块/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1202:in`each'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1202:in`migrate_without_lock'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1150:in`block in migration'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1319:in`with_advisory_lock'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1150:in'migrate'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1006:in'up'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:984:in'migrate'/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/tasks/database_tasks.rb:163:在'migrate'中/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/railties/databases.rake:58:在/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in`'/home/deploy/.rbenv/versions/2.3.1/bin/bundle:23:in`load'/home/deploy/.rbenv/versions/2.3.1/bin/bundle:23:in` elasticsearch上没有索引文件，默认模板上没有关于此设置的任何信息
我遇到了类似的问题，下面的错误消息清楚地解释了该问题.
[400]{"error":{"root_cause":[{"type":"illegal_argument_exception"，"reason":"TheNGram令牌生成器中的max_gram和min_gram之间的差异必须为小于或等于: 1 但是是[49].可以通过以下方式设置此限制更改[index.max_ngram_diff]索引级别设置.}]，"类型:" illegal_argument_exception，"原因:"NGram令牌生成器中的max_gram和min_gram之间的差异必须为小于或等于: 1 但是是[49].可以通过以下方式设置此限制更改[index.max_ngram_diff]索引级别设置.}，"状态:400}
基本上，默认情况下，NGram Tokenizer中的max_gram和min_gram之差不能超过1，并且如果您要更改此值，则在索引设置中，您需要通过添加以下设置来进行更改.
"max_ngram_diff":"50"->您可以根据需要提及此号码. 下面是我的索引设置，您可以在其中看到我的 max_gram 和 min_gram 的区别是 47 ，因此设置了max_ngram_diff 到 50 .
{设置":{指数": {分析": {分析器":{字首": {"type":"custom"，筛选": ["lowercaseFilter"]，"tokenizer":"edgeNGramTokenizer"}}，"tokenizer":{"edgeNGramTokenizer":{"token_chars":[信件"，数字"]，"min_gram":"1"，"type":"edgeNGram"，"max_gram":"40"}，"loginNGram":{"type":"nGram"，"min_gram":"3"，"max_gram":"50"}}}，"number_of_shards":"1"，"number_of_replicas":"0"，"max_ngram_diff":"50"}}} 编辑:添加官方弹性文档，该文档解释说max_gram的默认长度为2，min_gram为1，因此它们之间的默认差不能大于1，因此是例外.然后是同一文档的代码段
索引级别设置index.max_ngram_diff控制最大值允许max_gram和min_gram之间的差异.
Im trying to deploy a Ruby on Rails app on a Ubuntu 16.04 EC2 server but is giving a error about the difference between max_gram and min_gram on Elasticsearch, i don't have any experience with Elasticsearch yet so im totally lost here and i need some guidance to do this and learn how to set it to avoid this problems in the future.
The first time i did the deploy there was a error refusing the connection to localhost:9200 so i had to check if the service was running and even check the firewall and at the end i had to do a clean install and configure everything on elasticsearch.yml and now is running and working but when i try to deploy again is giving me a error, did a lot of search on internet there is a lot of documentation but i still don't get where to set these values.
This is the error im getting on the log:
-----> Migrating database...
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
[400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The difference between max_gram and min_gram in NGram Tokenizer must be less than or equal to: [1] but was [49]. This limit can be set by changing the [index.max_ngram_diff] index level setting."}],"type":"illegal_argument_exception","reason":"The difference between max_gram and min_gram in NGram Tokenizer must be less than or equal to: [1] but was [49]. This limit can be set by changing the [index.max_ngram_diff] index level setting."},"status":400}
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/base.rb:205:in `__raise_transport_error'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/base.rb:323:in `perform_request'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/transport/http/faraday.rb:20:in `perform_request'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-transport-6.0.2/lib/elasticsearch/transport/client.rb:131:in `perform_request'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api/namespace/common.rb:21:in `perform_request'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/elasticsearch-api-6.0.2/lib/elasticsearch/api/actions/indices/create.rb:86:in `create'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:16:in `create'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:203:in `create_index'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:270:in `reindex_scope'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/index.rb:196:in `reindex'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/searchkick-3.0.2/lib/searchkick/model.rb:59:in `searchkick_reindex'
/home/deploy/catalogindustry/releases/20190807135404/db/migrate/20180405153226_validated_true.rb:4:in `change'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:789:in `exec_migration'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:773:in `block (2 levels) in migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:772:in `block in migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/connection_adapters/abstract/connection_pool.rb:398:in `with_connection'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:771:in `migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:951:in `migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1232:in `block in execute_migration_in_transaction'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1302:in `ddl_transaction'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1231:in `execute_migration_in_transaction'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1203:in `block in migrate_without_lock'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1202:in `each'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1202:in `migrate_without_lock'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1150:in `block in migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1319:in `with_advisory_lock'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1150:in `migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:1006:in `up'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/migration.rb:984:in `migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/tasks/database_tasks.rb:163:in `migrate'
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/activerecord-5.0.7/lib/active_record/railties/databases.rake:58:in `block (2 levels) in '
/home/deploy/catalogindustry/shared/bundle/ruby/2.3.0/gems/rake-12.3.1/exe/rake:27:in `'
/home/deploy/.rbenv/versions/2.3.1/bin/bundle:23:in `load'
/home/deploy/.rbenv/versions/2.3.1/bin/bundle:23:in `
There is no index files on elasticsearch and there is nothing about this setting on the default template
I have faced a similar issue and below error message is clearly explaining the issue.
[400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The difference between max_gram and min_gram in NGram Tokenizer must be less than or equal to: 1 but was [49]. This limit can be set by changing the [index.max_ngram_diff] index level setting."}],"type":"illegal_argument_exception","reason":"The difference between max_gram and min_gram in NGram Tokenizer must be less than or equal to: 1 but was [49]. This limit can be set by changing the [index.max_ngram_diff] index level setting."},"status":400}
Basically, by Default, the difference between max_gram and min_gram in NGram Tokenizer can't be more than 1 and if you want you to change this, then in your index settings you need to change it by adding below setting.
"max_ngram_diff" : "50" --> you can mention this number accoding to your requirement.
Below is my index settings, where you can see I have a difference of 47 in my max_gram and min_gram hence set max_ngram_diff to 50.
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"prefix": {
"type": "custom",
"filter": [
"lowercaseFilter"
],
"tokenizer": "edgeNGramTokenizer"
}
},
"tokenizer": {
"edgeNGramTokenizer": {
"token_chars": [
"letter",
"digit"
],
"min_gram": "1",
"type": "edgeNGram",
"max_gram": "40"
},
"loginNGram": {
"type": "nGram",
"min_gram": "3",
"max_gram": "50"
}
}
},
"number_of_shards": "1",
"number_of_replicas": "0",
"max_ngram_diff" : "50"
}
}
}
Edit: Adding an official Elastic documentation, which explains that default length of max_gram is 2 and min_gram is 1, hence default difference between these can't be more than 1, hence the exception. And then snippet from the same doc
The index level setting index.max_ngram_diff controls the maximum allowed difference between max_gram and min_gram.
这篇关于尝试在Elasticsearch中设置max_gram和min_gram的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
