通过可靠的字典搜索 [英] Search through Reliable Dictionary
问题描述
我需要通过系统中的所有用户来创建Web API搜索功能. 客户端(使用电话)使用端点向我发送请求:
I need to create Web API search functionality through all users in my system. Client (using phone) sends me requests using endpoint:
HTTP 1.1 GET http://sf.cluster:80/
Path /search/users?q=Aa&take=10
其中 q 是在搜索字段中输入的字符串用户. 获取-手机要显示多少条目.
Where q is a string user entered in a search field. take - how much entries phone wants to show.
我从Azure存储表中上传了我的Reliable Dictionary 89000项目.它具有以下结构:
I uploaded in my Reliable Dictionary 89000 items from Azure Storage Table. It has structure:
IReliableDictionary<Guid, string>
我的搜索方法如下:
public async Task<IEnumerable<UserInfo>> Search(string q, int take)
{
var usersDictionary = await GetUsersDictionary();
IEnumerable<UserInfo> results;
using (var tx = StateManager.CreateTransaction())
{
var searchResults = (from r in (await usersDictionary.CreateEnumerableAsync(tx)).ToEnumerable()
where r.Value.StartsWith(q, StringComparison.InvariantCultureIgnoreCase)
select new UserInfo()
{
Id = r.Key,
Name = r.Value
}).Take(take);
results = new List<UserInfo>(searchResults);
await tx.CommitAsync();
}
return results;
}
问题:在电话上效果很好，我得到了预期的结果.但是，当我开始使用一堆请求(使用 Soap UI )工具同时推送约60个线程时， )，超时开始从1秒增加到35秒！看来我在某个地方犯了错误，或者选择了错误的搜索实现方式.
The problem: It works nice on the phone, I got what I expected. But when I have started push my endpoint with a bunch of requests (approx. ~60 threads simultaneously using Soap UI tool), a timeout started goes up from 1s to 35s! It looks like I made a mistake somewhere or choose a wrong way of search implementation.
有人实现了这样的功能吗?有人可以提供正确搜索方法的帮助吗?
Had somebody implemented some functionality like this? Can anybody help with correct search approach?
UPD:实现了无状态服务，我在其中存储具有名称的List<string>并对其进行相同的操作(在列表中搜索).结果:150-300ms.看来我应该将List存储在一个状态中(在有状态服务中)，并根据请求获取它.
UPD: Implemented stateless service where I store List<string> with names and do the same things their (search through the list). Results: 150-300ms. It looks like I should store List in a state (in stateful service) and get it upon request..
推荐答案
我不确定您的ToEnumerable方法的实现是什么，但是我所看到的大多数都是采用异步的相当懒惰的实现.可枚举并将其复制到列表中.现在，有了可靠的890,000个元素的字典，那是相当低效的.同样，事务就像互斥量一样，因此在复制此庞大列表时，您将锁定基础集合.我建议在此库中检查AsyncEnumerable linq实现，因为它实现了使用linq的有效方法服务结构AsyncEnumerable.使用它，您的搜索将看起来像这样:
I'm not sure what the implementation of your ToEnumerable method is, but most that I've seen are a fairly lazy implementation of just taking the async enumerable and copying it to a list. Now, with a reliable dictionary of 890,000 elems, that's quite inefficient. Also, transactions act like a mutex, so while you're copying this huge list you're locking the underlying collection. I would suggest checking out the AsyncEnumerable linq implementation in this library as it implements an efficient way to use linq with service fabrics AsyncEnumerable. Using that, your search would look something like this:
using (var tx = StateManager.CreateTransaction())
{
var enumerable = await usersDictionary.CreateEnumerableAsync(tx);
results = await enumerable.Where(kvp=>kvp.Value.StartsWith(q, StringComparison.InvariantCultureIgnoreCase))
.Select(kvp=> new UserInfo()
{
Id = r.Key,
Name = r.Value
})
.Take(take)
.ToListAsync(tx);
}
此外，作为旁注，由于您无需以任何方式修改基础集合，因此无需提交事务.提交事务只是告诉状态管理器您已经修改了状态并且已经完成更改的一种方式，然后它将更改后的值传播到第二级.如果这是重读状态，您甚至可以在辅助服务器上调用此方法，但请注意，写入可能尚未传播.
Also, as a side note, since you're not modifying the underlying collection in any way you don't need to commit the transaction. Committing the transaction is simply a way to tell the state manager that you have modified the state and you've finished making your changes, which it will then propagate the changed values to the secondaries. You could even call this method on the secondaries if this a read-heavy piece of state, but note that writes may not be propagated yet.
这篇关于通过可靠的字典搜索的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
