如何在 C# 中有效地并行执行多个异步调用? [英] How to execute multiple async calls in parallel efficiently in C#?

问题描述

我有一个 client id 列表，对于每个 client id，我需要从 cassandra 获取数据.因此，我并行执行所有这些客户端 ID，而不是使用对性能不利的 IN 子句查询.

I have a list of client id and for each client id I need to get data from cassandra. So I am executing all those client id's in parallel instead of using IN clause query which is not good for performance.

所以我想出了下面的代码，它为每个客户端 ID 执行多个异步调用，它通过从 cassandra 中获取数据来完成这项工作，但它是并行执行多个异步调用的正确方法还是我在这里做错了什么哪些会影响我的表现?

So I came up with below code which execute multiple async calls for each client id and it does the job by getting data out of cassandra but is it the right way to execute multiple async calls in parallel or am I doing something wrong here which can affect my performance?

public async Task<IList<Item>> GetAsync(IList<int> clientIds, int processId, int proc, Kyte kt)
{
    var clientMaps = await ProcessCassQueries(clientIds, (ct, batch) => mapper.SingleOrDefaultAsync<ItemMapPoco>(itemMapStmt, batch), "GetPIMValue");

    if (clientMaps == null || clientMaps.Count <= 0)
    {
        return null;
    }
    // .. do other stuff and return
}

// this executes multiple client ids in parallel - but is it the right way considering performance?
private async Task<List<T>> ProcessCassQueries<T>(IList<int> ids, Func<CancellationToken, int, Task<T>> mapperFunc, string msg) where T : class
{
    var requestTasks = ids.Select(id => ProcessCassQuery(ct => mapperFunc(ct, id), msg));
    return (await Task.WhenAll(requestTasks)).Where(e => e != null).ToList();
}

// this might not be good
private Task<T> ProcessCassQuery<T>(Func<CancellationToken, Task<T>> requestExecuter, string msg) where T : class
{
    return requestExecuter(CancellationToken.None);
}

我最近开始使用 C#，所以我对这方面的知识有限，所以我的代码在性能方面可能不太好.特别是 ProcessCassQueries 和 ProcessCassQuery 方法.考虑到它是产品代码，有什么可以在这里改进或可以用更好的方式编写的吗?

I recently started using C# so have limited knowledge around that so maybe my code might be not good in terms of performance. Specially ProcessCassQueries and ProcessCassQuery methods. Anything that can be improved here or can be written in a better way considering it's a prod code?

更新:

基于建议，使用信号量来限制异步调用的次数，如下所示:

Basis on suggestion, using semaphore to limit number of async calls as shown below:

private var semaphore = new SemaphoreSlim(20);

private async Task<List<T>> ProcessCassQueries<T>(IList<int> ids, Func<CancellationToken, int, Task<T>> mapperFunc, string msg) where T : class
{
    var tasks = ids.Select(async id => 
    {
        await semaphore.WaitAsync();
        try
        {
            return await ProcessCassQuery(ct => mapperFunc(ct, id), msg);
        }
        finally
        {
            semaphore.Release();
        }
    });

  return (await Task.WhenAll(tasks)).Where(e => e != null).ToList();
}

推荐答案

你的做法是正确的.您同时启动一堆任务，然后等待所有任务完成.这个特定的 C# 代码没有效率低下或瓶颈.你传递了一个硬编码的 CancellationToken.None 在 ProcessCassQuery 中，但不会影响性能.整个操作的性能现在取决于 Cassandra 数据库的行为，当它受到多个同时请求的轰炸时.如果它针对这种用途进行了优化，那么一切都会好起来的.如果不是，那么您当前的设置不能灵活地将并发级别配置为特定数据库引擎的最佳值.有关限制并发异步 I/O 操作数量的方法，请查看 这里.

What you are doing is correct. You are launching a bunch of tasks all at once, and then await all of them to complete. There is no inefficiency or bottleneck regarding this specific C# code. It is a bit strange that you pass a hardcoded CancellationToken.None in the ProcessCassQuery, but it will not affect the performance. The performance of the whole operation now depends on the behavior of the Cassandra database, when it is bombarded with multiple simultaneous requests. If it is optimized for this kind of usage then everything will be OK. If not, then your current setup doesn't offer the flexibility of configuring the level of concurrency to a value optimal for the specific database engine. For ways to limit the amount of concurrent async I/O operations look here.

作为旁注，根据官方 指南 异步方法 ProcessCassQueries 和 ProcessCassQuery 应该具有 Async 后缀.

As a side note, according to the official guidelines the asynchronous methods ProcessCassQueries and ProcessCassQuery should have the Async suffix.

这篇关于如何在 C# 中有效地并行执行多个异步调用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！