为什么Flurl.Http DownloadFileAsync/Http客户端GetAsync需要这么长时间才能取消任务 [英] Why task cancellation takes so long in Flurl.Http DownloadFileAsync / HttpClient GetAsync

问题描述

在编写从服务器并行下载图像的类时(使用由DataFlow TPL库支持的消费者/生产者模式)，使用ActionBlock和Flurl.Http工具方法DownloadFileAsync，我已经意识到取消操作需要很长时间。由于所有下载操作都共享一个CancellationToken，我预计所有任务都会立即(或几乎)取消。实际上，如果我产生大量并行下载，取消所有任务/线程可能需要几分钟时间。我已经解决了这个问题，将ExecutionDataflowBlockOptions.MaxDegreeOfParallelism属性设置为10。这样，在任何给定时间最多有10个要取消的并发下载(这仍然不是我预期的立即操作)。

我已经创建了一个控制台.NET5程序，该程序隔离地重现该问题(没有数据流TPL、ActionBlock等)。它首先询问并发下载的数量(默认情况下按Enter键：100次下载)。然后，它使用Flurl.Http(使用HttpClient)并行生成所有这些下载，并向每个操作传递一个CancellationToken。然后等待按键，然后通过调用CancellationTokenSource.Cancel方法取消挂起的下载。最后，它会打印一些统计数据，包括成功和失败/取消的下载次数，以及完成取消所需的时间。以下是完整的代码：

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
using Flurl.Http;

const string imageSource = "https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png";
const int defaultCount = 100;
var watch = Stopwatch.StartNew();
int completed = 0;
int failed = 0;

Console.WriteLine("Flurl.DownloadFileAsync Cancellation test!");
Console.Write($"Number of downloads ({defaultCount}): ");
var input = Console.ReadLine();
if (!int.TryParse(input, out var count)) 
    count = defaultCount;
Console.WriteLine($"Will spawn {count} parallel downloads of '{imageSource}'");

CancellationTokenSource cts = new CancellationTokenSource();
List<Task> tasks = new();
for (int i = 0; i < count; i++) 
    tasks.Add(Download(i));

Console.WriteLine("Hit anything to cancel...");
Console.ReadKey(true);

log("Cancelling pending downloads");
var cancelMoment = watch.Elapsed;
cts.Cancel();
Task.WaitAll(tasks.ToArray());
log("Downloads cancelled. Program ended!");

Console.WriteLine($"### Total cancellation time: {watch.Elapsed - cancelMoment} -> Completed: {completed}, Failed/Cancelled: {failed}");

async Task Download(int i) {
    var fn = $"test_{i}.png";
    try {
        await imageSource.DownloadFileAsync(fn, cancellationToken: cts.Token);
        Interlocked.Increment(ref completed);
        log($"DONE: {fn}");
    } catch(Exception e) {
        Interlocked.Increment(ref failed);
        log($"# ERROR: {fn}/r/n >> {e.Message}");
    }
}

void log(string s) => Console.WriteLine($"{watch.Elapsed}- {s}");

最让我印象深刻的是，允许所有下载完成(即使我输入了1000次下载)比取消操作更快。我不知道是否存在任何类型的死锁(这会导致操作在锁定超时后结束)，或者这些下载的取消是否只是中断。我找不到对此问题的好解释或解决方案。

要重现该问题，您必须在所有下载完成之前按某个键以取消挂起的下载。如果你选择了正确的时机，你可以让几次下载成功。如果你点击太快，你会取消所有下载。如果您等待的时间太长，则所有下载将已完成。

此运行导致以下结果：

取消99个挂起的操作花费了55秒以上的时间。如果我只允许完成所有下载，则所需时间比取消相同操作所需的时间要少得多。

更新

---

我已经完全删除了Flurl，并直接使用HttpClient下载文件，但问题仍然存在。我已将Downlod方法更改为以下内容：

async Task Download(int i) {
    var fn = $"test_{i}.png";
    try {
        var r = await client.GetAsync(imageSource, cancellationToken: cts.Token);
        using var httpStm = await r.Content.ReadAsStreamAsync(cts.Token);
        var file = new FileInfo(fn);
        using var f = file.OpenWrite();
        await httpStm.CopyToAsync(f, cts.Token);
        Interlocked.Increment(ref completed);
        log($"DONE: {fn}");
    } catch(Exception e) {
        Interlocked.Increment(ref failed);
        log($"# ERROR: {fn}/r/n >> {e.Message}");
    }
}

结果与基于FLURL的实现相同(毕竟，Flurl.Http只是HttpClient的包装)。

更新2

---

我已经将下载方法更改为简单地等待可取消的Task.Delay，100个操作的取消时间现在约为2秒。虽然它更快，但它不是即时的，从屏幕上日志的计时来看，在我看来，取消是顺序触发的，而不是并行/立即触发的。此下载的代码为：

async Task Download(int i) {
    var fn = $"test_{i}.png";
    try {
        await Task.Delay(TimeSpan.FromMinutes(1), cts.Token);
        Interlocked.Increment(ref completed);
        log($"DONE: {fn}");
    } catch (Exception e) {
        Interlocked.Increment(ref failed);
        log($"# ERROR: {fn}/r/n >> {e.Message}");
    }
}

以下屏幕截图显示了上述代码的结果：

有人对此有好的解释或解决方案吗？

推荐答案

如果您将取消令牌单独传递给任务而不是父任务Task.WaitAll()，则在您调用cts.Cancel()之后，其他任务将不知道取消请求，直到为每个任务调用Download()。

将令牌传递给父级，使其充当所有任务的协调器：

Task.WaitAll(tasks.ToArray(), cts.Token);

然后，一旦触发取消，将不需要执行剩余的任务，从而为您节省取消时间。

Flurl.DownloadFileAsync Cancellation test!
Number of downloads (100):
Will spawn 100 parallel downloads of 'https://www.google.com/images/branding/googlelogo/1x/googlelogo_color_272x92dp.png'
Hit anything to cancel...
00:00:03.8603945- DONE: test_3.png
00:00:03.8603937- DONE: test_41.png
00:00:03.8603967- DONE: test_12.png
00:00:03.8603920- DONE: test_22.png
00:00:03.8640374- DONE: test_27.png
00:00:03.8651895- DONE: test_57.png
00:00:03.8652114- DONE: test_64.png
00:00:03.8652039- DONE: test_1.png
00:00:03.8652149- DONE: test_10.png
00:00:03.8652222- DONE: test_0.png
00:00:03.8653910- DONE: test_2.png
00:00:03.8865928- DONE: test_28.png
00:00:03.8990767- DONE: test_11.png
00:00:03.8993285- DONE: test_20.png
00:00:03.9048105- DONE: test_6.png
00:00:03.9056048- DONE: test_68.png
00:00:03.9067027- DONE: test_69.png
00:00:03.9075434- DONE: test_54.png
00:00:03.9094678- DONE: test_5.png
00:00:03.9166961- DONE: test_50.png
00:00:03.9169709- DONE: test_13.png
00:00:03.9185492- DONE: test_87.png
00:00:03.9298495- DONE: test_7.png
00:00:03.9326280- DONE: test_43.png
00:00:03.9327424- DONE: test_33.png
00:00:03.9357554- DONE: test_45.png
00:00:03.9363720- DONE: test_37.png
00:00:03.9386855- DONE: test_30.png
00:00:03.9393222- DONE: test_95.png
00:00:03.9427672- DONE: test_14.png
00:00:03.9463875- DONE: test_84.png
00:00:03.9487063- DONE: test_34.png
00:00:03.9488809- DONE: test_80.png
00:00:03.9493661- DONE: test_90.png
00:00:03.9533510- DONE: test_17.png
00:00:03.9550988- DONE: test_85.png
00:00:03.9559576- DONE: test_71.png
00:00:03.9559739- DONE: test_31.png
00:00:03.9581015- DONE: test_52.png
00:00:03.9595559- DONE: test_89.png
00:00:03.9596396- DONE: test_61.png
00:00:03.9604608- DONE: test_36.png
00:00:03.9605789- DONE: test_26.png
00:00:03.9643205- DONE: test_25.png
00:00:03.9708155- DONE: test_18.png
00:00:03.9712862- DONE: test_39.png
00:00:03.9780414- DONE: test_98.png
00:00:03.9782002- DONE: test_42.png
00:00:03.9788898- DONE: test_48.png
00:00:03.9844171- DONE: test_15.png
00:00:03.9856963- DONE: test_16.png
00:00:03.9862520- DONE: test_96.png
00:00:03.9923971- DONE: test_58.png
00:00:03.9944836- DONE: test_59.png
00:00:04.0070565- DONE: test_40.png
00:00:04.0093207- DONE: test_29.png
00:00:04.0119729- DONE: test_55.png
00:00:04.0148981- DONE: test_24.png
00:00:04.0163724- DONE: test_32.png
00:00:04.0173879- DONE: test_9.png
00:00:04.0191403- DONE: test_46.png
00:00:04.0426426- DONE: test_23.png
00:00:04.0549689- DONE: test_81.png
00:00:04.0550977- DONE: test_4.png
00:00:04.0554659- DONE: test_63.png
00:00:04.1206750- DONE: test_21.png
00:00:04.1207557- DONE: test_44.png
00:00:04.1773007- DONE: test_65.png
00:00:04.1854760- DONE: test_62.png
00:00:04.1854846- DONE: test_38.png
00:00:04.1883149- Cancelling pending downloads
00:00:04.2348932- # ERROR: test_19.png/r/n >> A task was canceled.
00:00:04.2757044- # ERROR: test_35.png/r/n >> A task was canceled.
00:00:04.2757328- # ERROR: test_8.png/r/n >> A task was canceled.
00:00:04.2759605- # ERROR: test_56.png/r/n >> A task was canceled.
00:00:04.2763499- Downloads cancelled. Program ended!
### Total cancellation time: 00:00:00.0954826 -> Completed: 70, Failed/Cancelled: 4
00:00:04.2845926- # ERROR: test_70.png/r/n >> A task was canceled.

请注意，取消时间要短得多，因为我们只需要取消当时正在执行的任务，而不是所有任务都会自己收到取消的通知。我们完成了70项任务，其中4项被取消，其余的刚刚放弃，取消时间大大缩短。

这篇关于为什么Flurl.Http DownloadFileAsync/Http客户端GetAsync需要这么长时间才能取消任务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！