Python Elasticseach索引错误 [英] Python Elasticseach indexing error

问题描述

Elasticsearch在今天之前运行良好且很好.

Elasticsearch was working well and fine before today.

某些文档无法建立索引并显示错误:

Some documents which are failing to index with error:

u'Limit of total fields [1000] in index [mintegral_incent] has been exceeded' 

错误:

"BulkIndexError: (u'14 document(s) failed to index.', [{u'index': {u'status': 400, u'_type': u'mintegral_incent', u'_id': u'168108082', u'error': {u'reason': u'Limit of total fields [1000] in index [mintegral_incent] has been exceeded', u'type': u'illegal_argument_exception'}

使用Amazon Elastic Service

Using Amazon Elastic service

Elasticsearch版本5.1

Elasticsearch Version 5.1

from elasticsearch import Elasticsearch
from elasticsearch import helpers
es_repo = Elasticsearch(hosts=[settings.ES_INDEX_URL],
                        verify_certs=True)

代码发出问题:

def bulk_index_offers(index_name, id_field, docs):
    actions = []
    for doc in docs:
        action = {
            "_index": index_name,
            "_type": index_name,
            "_id": doc.get(id_field),
            "_source": doc
        }
        actions.append(action)
    # Error at this following line.
    resp = helpers.bulk(es_repo, actions)
    return resp

我尝试过的事情:

我尝试将块设置为较小，并将read_timeout从默认的10增加到30像这样: resp = helpers.bulk(es_repo，actions，chunks = 500，read_timeout = 30)

但仍然面临着同样的问题.

But still facing same issue.

请帮助.

推荐答案

默认情况下，映射类型仅允许

By default, a mapping type is only allowed to contain up to 1000 fields and it seems you are exceeding that limit. In order to increase that threashold you can run this command:

PUT mintegral_incent/_settings
{ 
  "index": {
    "mapping": {
      "total_fields": {
        "limit": "2000"
      }
    }
  }
}

使用卷曲，看起来像这样:

Using curl, it'd look like this:

curl -XPUT http://<your.amazon.host>/mintegral_incent/_settings -d '{ 
  "index": {
    "mapping": {
      "total_fields": {
        "limit": "2000"
      }
    }
  }
}'

然后，您可以再次运行您的批量脚本，它应该可以工作.

Then you can run your bulk script again and it should work.

这篇关于Python Elasticseach索引错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！