在 Solr 中，“文本"的最大大小是多少?场地? [英] In Solr, what is the maximum size of a "text" field?

问题描述

在您的应用中使用 Solr 客户端时，text 多行字段的最大大小是多少?

When using Solr client in your app, what is the max size of a text multi line field?

我可以将大型 xml 文档作为文本发送吗?

Can I send huge xml documents as text?

例如

SolrInputDocument document = new SolrInputDocument();
document.addField("id", rec.getId());
document.addField("hugeTextFile_txt", hugeTextFile);        
UpdateResponse response = solr.add(document);
solr.commit();  

推荐答案

更新

我使用 text fieldType 使用了相同的单元测试.下面是我使用的声明.请注意，我已从声明中删除了分析器部分.

I used the same unit test using text fieldType. Below is the declaration I used. Please note that I have removed analyzer section from declaration.

<fieldType name="text" class="solr.TextField"/>

我能够添加 500,000,000 个字符并成功将其编入索引.为了获得更高的价值，我得到了 Java 堆空间 错误，这与 solr 无关.

I was able to add 500,000,000 characters and index it successfully. For higher value I got Java heap space error, which is not related to the solr.

我试图通过向字段添加一个大值来执行一个简单的测试.我发现的限制是 32,766 字节.之后它会抛出IllegalArgumentException.email 的 fieldType 是 string.

I tried to perform a simple test by adding a large value to a field. The limit I found is 32,766 bytes. After that It throws IllegalArgumentException. The fieldType for email was string.

<fieldType name="string" class="solr.StrField" sortMissingLast="true" />

@Test
public void test() throws IOException, SolrServerException {
  SolrInputDocument document = new SolrInputDocument();
  document.addField("profileId", TestConstants.PROFILE_ID);
  StringBuilder builder = new StringBuilder();
  for (int i = 0; i<32767; i++) {
    builder.append((char)((i%26)+'a'));
  }
  document.addField("email", builder.toString());
  solrClient.add(document);
  solrClient.commit();
}

上面为 32767 及更多抛出的异常:

Exception thrown by above for 32767 and more:

Caused by: java.lang.IllegalArgumentException: 文档在 field="email" 中至少包含一个巨大的术语(其 UTF8 编码比最大长度 32766 长)，所有这些都被跳过.请更正分析器以不产生此类术语.第一个大项的前缀是:'[97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 15,114, 117, 118, 119, 120, 121, 122, 97, 98, 99, 100]...', 原始消息:字节长度最多为32766；得到 32767

Caused by: java.lang.IllegalArgumentException: Document contains at least one immense term in field="email" (whose UTF8 encoding is longer than the max length 32766), all of which were skipped. Please correct the analyzer to not produce such terms. The prefix of the first immense term is: '[97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 97, 98, 99, 100]...', original message: bytes can be at most 32766 in length; got 32767

我希望这会有所帮助.

这篇关于在 Solr 中，“文本"的最大大小是多少?场地?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！