SOLR指数大小减少 [英] SOLR index size reduction

问题描述

我们为一个大型项目提供了一些大规模的SOLR指数，其消耗的空间超过50 GB。

We have a some massive SOLR indices for a large project, and its consuming above 50 GB of space .

我们已经考虑了几种减少与更改指数内容相关的尺寸的方法，但我很好奇是否可能有任何变化我们可以制作SOLR索引，将其大小减小2个数量级或更多，这与（1）我们可以运行的维护命令或（2）可能无法正确设置的简单配置参数直接相关。

We have considered several ways to reduce the size that are related to changing the content in the indices, but I am curious of wether or not there might be any changes we can make to a SOLR index which will reduce its size by 2 orders of magnitude or more, which are directly related to either (1) maintainance commands we can run or (2) simple configuration parameters which may not be set right.

另一个相关的问题是（3）是否有办法在SOLR内部交易索引大小以获得性能，如果有，它将如何运作？

Another relevant question is (3) Is there a way to trade index size for performance inside of SOLR, and if so , how would it work ?

对此的任何想法都将不胜感激...谢谢！

Any thoughts on this would be appreciated... Thanks!

推荐答案

您可以做一些事情来交易索引大小的性能。例如，整数（int）字段使用的空间小于trie整数（tint），但使用int时范围查询会更慢。

There are a couple things you might be able to do to trade performance for index size. For example, an integer (int) field uses less space than a trie integer (tint), but range queries will be slower when using an int.

进行大幅缩减在你的索引中，你几乎肯定需要仔细查看你正在使用的字段。

To make major reductions in your index, you will almost certainly need to look more closely at the fields you are using.

• 你是否使用了大量的存储字段？如果是这样，尝试从索引中删除存储的字段，并在从Solr获得结果后查询数据库以获取必要的数据。


• 将omitNorms =true添加到文本字段不需要长度标准化


• 将omitPositions =true添加到不需要词组匹配的文本字段


• 特殊字段，如NGrams ，可占用大量空间


• 您是否从文本字段中删除停用词？

• Are you using a lot of stored fields? If so, try removing the stored fields from the index and query your database for the necessary data once you've got the results back from Solr.
• Add omitNorms="true" to text fields that don't need length normalization
• Add omitPositions="true" to text fields that don't require phrase matching
• Special fields, like NGrams, can take up a lot of space
• Are you removing stop words from text fields?

这篇关于SOLR指数大小减少的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！