具有100万个键的服务结构可靠的字典性能 [英] Service fabric reliable dictionary performance with 1 million keys

问题描述

我正在评估带有大约100万个密钥的可靠字典的Service Fabric的性能.我得到的结果令人非常失望，所以我想检查我的代码或期望是否错误.

I am evaluating the performance of Service Fabric with a Reliable Dictionary of ~1 million keys. I'm getting fairly disappointing results, so I wanted to check if either my code or my expectations are wrong.

我有一个用初始化的字典 dict = await _stateManager.GetOrAddAsync<IReliableDictionary2<string, string>>("test_"+id);

I have a dictionary initialized with dict = await _stateManager.GetOrAddAsync<IReliableDictionary2<string, string>>("test_"+id);

id对于每个测试运行都是唯一的.

id is unique for each test run.

我用字符串列表填充它，例如 "1-1-1-1-1-1-1-1-1"， "1-1-1-1-1-1-1-1-2"， "1-1-1-1-1-1-1-1-3" ....最多576,000个项目.字典中的值未使用，我目前仅使用"1".

I populate it with a list of strings, like "1-1-1-1-1-1-1-1-1", "1-1-1-1-1-1-1-1-2", "1-1-1-1-1-1-1-1-3".... up to 576,000 items. The value in the dictionary is not used, I'm currently just using "1".

将所有项目添加到词典大约需要3分钟.我必须一次将交易拆分为100,000，否则该交易将永远挂起(在您需要CommitAsync()之前，交易中的操作数量是否有限制?)

It takes about 3 minutes to add all the items to the dictionary. I have to split the transaction to 100,000 at a time, otherwise it seems to hang forever (is there a limit to the number of operations in a transaction before you need to CommitAsync()?)

//take100_000 is the next 100_000 in the original list of 576,000
using (var tx = _stateManager.CreateTransaction())
{
    foreach (var tick in take100_000) {
        await dict.AddAsync(tx, tick, "1");
    }
    await tx.CommitAsync();
}

在那之后，我需要遍历字典以访问每个项目:

After that, I need to iterate through the dictionary to visit each item:

using (var tx = _stateManager.CreateTransaction())
{

    var enumerator = (await dict.CreateEnumerableAsync(tx)).GetAsyncEnumerator();

    try
    {
        while (await enumerator.MoveNextAsync(ct))
        {
            var tick = enumerator.Current.Key;                
            //do something with tick                    
        }
    }
    catch (Exception ex)
    {
        throw ex;
    }
}

这需要16秒.

我不太担心写入时间，我知道它必须被复制并持久化.但是，为什么要花这么长时间阅读? 576,000个17个字符的字符串键在内存中的长度不应超过11.5mb，并且值仅是单个字符，将被忽略.可靠的集合不是缓存在ram中吗?要遍历相同值的常规Dictionary需要13ms.

I'm not so concerned about the write time, I know it has to be replicated and persisted. But why does it take so long to read? 576,000 17-character string keys should be no more than 11.5mb in memory, and the values are only a single character and are ignored. Aren't Reliable Collections cached in ram? To iterate through a regular Dictionary of the same values takes 13ms.

然后我在一本空字典中(一次交易)调用了ContainsKeyAsync 576,000次.这花费了112秒.尝试在其他任何数据结构上进行此操作大约需要0毫秒.

I then called ContainsKeyAsync 576,000 times on an empty dictionary (in 1 transaction). This took 112 seconds. Trying this on probably any other data structure would take ~0 ms.

这是在本地1节点群集上.部署到Azure时，我得到了类似的结果.

This is on a local 1 node cluster. I got similar results when deployed to Azure.

这些结果合理吗?我应该检查任何配置吗?我是在做错什么，还是我的期望很不准确?如果是这样，是否有更适合这些要求的东西? (约一百万个小键，无值，持续的事务更新)

Are these results plausible? Any configuration I should check? Am I doing something wrong, or are my expectations wildly inaccurate? If so, is there something better suited to these requirements? (~1 million tiny keys, no values, persistent transactional updates)

推荐答案

好吧，值得一试:

• 并非所有内容都存储在内存中.为支持大型可靠集合，某些值被缓存，某些值驻留在磁盘上，这可能会导致在检索值时产生额外的I/O.您要求的数据.我听说有传言说在某个时候我们可能有机会调整缓存策略，但是我认为它尚未实现.

• Not everything is stored in memory. To support large Reliable Collections, some values are cached and some of them reside on disk, which potentially could lead to extra I/O while retrieving the data you request. I've heard a rumor that at some point we may get a chance to adjust the caching policy, but I don't think it has been implemented already.

您一个接一个地遍历数据读取记录.恕我直言，如果您尝试对任何数据源发出半百万个单独的顺序查询，结果将不会很乐观.我并不是说每个MoveNext()都会导致一个单独的I/O操作，但我想说的是总体上它看起来并不像是单个提取.

You iterate through the data reading records one by one. IMHO, if you try to issue half a million separate sequential queries against any data source, the outcome won't be much optimistic. I'm not saying that every single MoveNext() results in a separate I/O operation, but I'd say that overall it doesn't look like a single fetch.

这取决于您拥有的资源.例如，尝试在具有单个分区和三个副本的本地计算机上重现您的案件，我平均获得了5秒的记录.

It depends on the resources you have. For instance, trying to reproduce your case on my local machine with a single partition and three replicas, I get the records in 5 seconds average.

关于变通办法，请注意以下几点:

Thinking about a workaround, here is what comes in mind:

• 分块我试图做同样的事情，将记录拆分为以10个元素为上限的字符串数组(IReliableDictionary< string，string []>).因此，基本上数据量是相同的，但是时间范围从5秒减少到7ms.我猜是否将项目保持在80KB以下，从而减少了往返次数并保持了LOH较小，您应该会看到性能有所改善.

• Chunking I've tried to do the same stuff splitting records into string arrays capped with 10 elements(IReliableDictionary< string, string[] >). So essentially it was the same amount of data, but the time range was reduced from 5sec down to 7ms. I guess if you keep your items below 80KB thus reducing the amount of round-trips and keeping LOH small, you should see your performance improved.

过滤

Filtering CreateEnumerableAsync has an overload that allows you to specify a delegate to avoid retrieving values from the disk for keys that do not match the filter.

希望这是有道理的.

这篇关于具有100万个键的服务结构可靠的字典性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！