为什么索引更新在Solr 4.10.x中花费了这么多时间？ [英] Why does an index update take so much time in Solr 4.10.x?

问题描述

我正在使用solr 4.10.3。我必须更新大约100,000个索引。查询类似于每个文档

I am using solr 4.10.3. I have to update about 100,000 indexes. Queries are like per document

curl 'localhost:8900/solr/update?commit=true' -H 'Content-type:application/json' -d '[{"id":"org.wikipedia.ur:http/wiki/%D9%85%DB%8C%D9%84","group":{"set":"wiki"}}]'

开始更新此索引后，经过12个小时后，只更新了48000个文档。

After starting to update this index, after 12 hours passed, only 48000 documents are updated.

问题出在哪里。任何人都可以指导我吗？

Where is the problem. Can anyone guide me?

推荐答案

您正在对每个curl请求使用硬提交。这会强制solr在每次提交时将段（用于存储索引的lucene数据结构）推送到磁盘。 Solr总是将数据写入新的段，看起来它迫使它创建100K段。

You are using hard commit with each curl request. This forces solr to push the segment (lucene data structure for storing the indexes) to disk with every commit. Solr always write the data into new segments and looks like it force it to create 100K segments.

Solr使用mergePolicy作为 TieredMergePolicy 和 mergeFactor 作为 10 每次solr有10个几乎相等大小的段时合并的默认参数。此合并过程使用 ConcurrentMergeScheduler 实现在后台运行。

Solr uses mergePolicy as TieredMergePolicy and mergeFactor as 10 default parameter which merge every time solr have 10 almost equal size segments. This merge processes runs in the background using with ConcurrentMergeScheduler implementation.

此合并过程是CPU密集型的。在这里，您可以使用 softCommit 而不是 hardCommit 。这可能对你有帮助。

This merge process is CPU intensive. Here you can use softCommit instead of hardCommit. That might help you.

这篇关于为什么索引更新在Solr 4.10.x中花费了这么多时间？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！