SOLR 索引大小减少 [英] SOLR index size reduction
问题描述
我们有一个大型项目的大量 SOLR 索引，它消耗了 50 GB 以上的空间.
We have a some massive SOLR indices for a large project, and its consuming above 50 GB of space .
我们已经考虑了几种减少与更改索引内容相关的大小的方法，但我很好奇我们是否可以对 SOLR 索引进行任何更改以将其大小减少 2数量级或更多，这与 (1) 我们可以运行的维护命令或 (2) 可能设置不正确的简单配置参数直接相关.
We have considered several ways to reduce the size that are related to changing the content in the indices, but I am curious of wether or not there might be any changes we can make to a SOLR index which will reduce its size by 2 orders of magnitude or more, which are directly related to either (1) maintainance commands we can run or (2) simple configuration parameters which may not be set right.
另一个相关问题是 (3) 有没有办法在 SOLR 内部用索引大小来换取性能，如果是这样，它将如何工作?
Another relevant question is (3) Is there a way to trade index size for performance inside of SOLR, and if so , how would it work ?
对此的任何想法将不胜感激...谢谢！
Any thoughts on this would be appreciated... Thanks!
推荐答案
您可能可以做一些事情来交换索引大小的性能.例如，整数 (int) 字段使用的空间比 trie 整数 (tint) 少，但使用 int 时范围查询会更慢.
There are a couple things you might be able to do to trade performance for index size. For example, an integer (int) field uses less space than a trie integer (tint), but range queries will be slower when using an int.
要大幅减少索引，您几乎肯定需要更仔细地查看正在使用的字段.
To make major reductions in your index, you will almost certainly need to look more closely at the fields you are using.
- 您是否使用了大量存储字段?如果是这样，请尝试从索引中删除存储的字段，并在从 Solr 获得结果后查询您的数据库以获取必要的数据.
- 将 omitNorms="true" 添加到不需要长度规范化的文本字段
- 将 omitPositions="true" 添加到不需要词组匹配的文本字段
- 特殊领域，如 NGrams，会占用大量空间
- 您是否要从文本字段中删除停用词?
这篇关于SOLR 索引大小减少的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
