使用NEST和C#进行弹性搜索滚动 [英] Elastic Search scrolling with NEST and C#

问题描述

我正在使用以下代码在弹性搜索框中循环/滚动所有文档:

I am using the following code to loop/scroll over all documents in my elastic search box:

const string indexName = "bla";
var client = GetClient(indexName);
const int scrollTimeout = 1000;

var initialResponse = client.Search<Document>
    (scr => scr.Index(indexName)
    .From(0)
    .Take(100)
    .MatchAll()
    .Scroll(scrollTimeout))
;

List<XYZ> results;
results = new List<XYZ>();

if (!initialResponse.IsValid || string.IsNullOrEmpty(initialResponse.ScrollId))
throw new Exception(initialResponse.ServerError.Error.Reason);

if (initialResponse.Documents.Any())
results.AddRange(initialResponse.Documents);

var scrollid = initialResponse.ScrollId;
bool isScrollSetHasData = true;
while (isScrollSetHasData)
{
    var loopingResponse = client.Scroll<XYZ>(scrollTimeout, scrollid);

    if (loopingResponse.IsValid)
    {
        results.AddRange(loopingResponse.Documents);
        scrollid = loopingResponse.ScrollId;
    }
    isScrollSetHasData = loopingResponse.Documents.Any();

    // do some amazing stuff
}

client.ClearScroll(new ClearScrollRequest(scrollid));

出于某种原因，loopingResponse为空的时间比预期的要早得多-即滚动完成.有人可以看到我的代码有根本性的错误吗?谢谢！

For some reason loopingResponse is empty much sooner than expected - i.e. the scroll finishes. Can someone see something fundamentally wrong with my code? Thanks!

推荐答案

查看您的代码，我认为 scrollTimeout 可能是问题.通常，滚动用于返回大块数据，而1000ms不足以使请求之间的搜索上下文保持活动状态.您可以尝试将其增加到几分钟，以找到最适合您的案例的数字:

Looking at your code I think scrollTimeout could be the problem. Usually scroll is used for big chunks of data to be returned and 1000ms is not enough to keep the search context alive between requests. You could try to increase it to several minutes to find the best number for your case:

var scrollTimeout = new Time(TimeSpan.FromMinutes(3));

或根据源代码您可以使用时间单位(微米，纳米，ms，s，m，h和d):

or according to source code you could use Time units (micros, nanos, ms, s, m, h, and d):

var response = client.Search<Document>(scr => scr.Index(indexName)
    ...
    .Scroll("3m")
    );

这篇关于使用NEST和C#进行弹性搜索滚动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！