从FTP下载文件仅在Azure Web应用程序中失败 [英] File download from FTP fails only in Azure web app

问题描述

我有这个非常基本的代码，可以从FTP服务器下载文本文件列表:

I have this very basic code that downloads a list of text files from an FTP server:

foreach (var fileUri in files)
{
    try
    {
        var ftpRequest = (FtpWebRequest)FtpWebRequest.Create(fileUri);
        ftpRequest.Method = WebRequestMethods.Ftp.DownloadFile;
        ftpRequest.Credentials = new NetworkCredential("***", "***");
        ftpRequest.UseBinary = false;
        ftpRequest.Timeout = 10_000;
        ftpRequest.ReadWriteTimeout = 10_000;

        using (WebResponse response = await ftpRequest.GetResponseAsync().ConfigureAwait(false))
        using (var stream = response.GetResponseStream())
        {
            // Copy the file to an Azure storage
            await CopyAsync(stream, container, GetFilename(fileUri), false).ConfigureAwait(false);
        }
    }
    catch (Exception e)
    {
        Logger.Error(Unknow, $"Error downloading {fileUri.AbsoluteUri}:\n{e}");
    }
}

从Visual Studio运行时，此代码可以完美运行. 但是，当在Azure中作为Web作业运行时，它只会下载几个文件，然后永久挂起.

When ran from Visual Studio, this code works perfectly. But when ran as a web job in Azure, it only downloads a couple of files before hanging forever.

我没有超时或异常.

我添加了一些调试日志，发现它卡在了GetResponseAsync()上. 我尝试了不使用ConfigureAwait(false)的情况，但行为却相同.

I added some debug logs and discovered that it's stuck on GetResponseAsync(). I tried without the ConfigureAwait(false) but got the same behaviour.

我还使用了此答案来检查网络跟踪，但所有FTP我在日志中找到的命令还可以.对于成功下载的每个文件，我都会看到以下几行，但是对于被阻止的文件，看不到任何不完整"的FTP命令:

I also used this answer to check the network traces but all FTP commands I found in the logs were OK. I see the following lines for every file successfully downloaded but don't see any "incomplete" FTP command for the file that's blocking:

System.Net Information: 0 : [22712] FtpWebRequest#2543959::.ctor(ftp://<ftpserver>/<filename>)
System.Net Information: 0 : [21728] FtpWebRequest#2543959::BeginGetResponse(Method=RETR.)
System.Net Information: 0 : [21728] Associating FtpWebRequest#2543959 with FtpControlStream#64735656
System.Net Information: 0 : [21728] FtpControlStream#64735656 - Sending command [PASV]
System.Net Information: 0 : [9444] FtpControlStream#64735656 - Received response [227 Entering Passive Mode (104,45,19,12,117,54).]
System.Net Information: 0 : [9444] FtpControlStream#64735656 - Sending command [RETR <filename>]
System.Net Information: 0 : [17608] FtpControlStream#64735656 - Received response [150 Opening ASCII mode data connection for <filename> (19262 bytes)]
System.Net Information: 0 : [21804] FtpControlStream#64735656 - Received response [226 Transfer complete]
System.Net Information: 0 : [21804] FtpWebRequest#2543959::(Releasing FTP connection#64735656.)

任何帮助/领导将不胜感激！

Any help/lead will be greatly appreciated!

编辑1

我尝试使用WebClient而不是FtpWebRequest并获得了相同的结果.它可以在我的机器上完美运行，但是当作为Web作业在Azure上运行时，它会下载几个文件，然后挂在webClient.DownloadFileTaskAsync(...)行上.另外，为了确保我上传到Azure存储不会干扰该过程，我现在只需在本地下载文件即可.再说一遍:没有例外，也没有超时.它只是永远挂着.

I tried using WebClient instead of FtpWebRequest and got the same kind of result. It works perfectly on my machine but when running on Azure as a web job, it downloads a couple of files, and then hangs on the webClient.DownloadFileTaskAsync(...) line. Also, just to be sure that my uploading to an Azure storage wasn't interfering in the process, I now simply download the files locally. And agin: no exception and no timeout. It just hangs forever.

编辑2

我尝试删除将文件上传到我的Azure存储的行，以防万一，但无济于事.然后，按照第一个评论者的建议，我添加了一个超时包装器"，该包装器实际上将我的代码作为任务运行Task.Delay(10_000)，并等待第一个任务完成.如果下载任务首先完成，那么一切都很好，但是如果Task.Delay(10_000)首先完成，我只是中止下载请求，然后重试.这样做时，我观察到以下情况:首先，第一次尝试都下载了所有文件.然后，某些文件需要进行2次尝试，并且有时即使经过10次尝试，所有文件都会失败.因此，基本上，一切开始都很好，但是很快就会降级.

I tried removing the line that uploads the file to my Azure storage, just in case, but to no avail. Then, following the first commenter's suggestion, I added a "timeout wrapper" that essentially runs my code as a task, a Task.Delay(10_000), and wait for the first task to finish. If the download task finishes first, then everything is good, but if the Task.Delay(10_000) finishes first, I simply abort the download request and try again. Doing this, I observe the following: at first, files are all downloaded at the first attempt. Then, some files need 2 attempts, and at some point, all files fail despite 10 attempts. So basically, everything starts well but quickly degrades.

推荐答案

很好...我觉得有些愚蠢，但是我猜想问题不在代码中.这只是CPU使用率的问题.最终，我发现(在Azure支持的帮助下)我在同一App Service Plan上还有其他Web作业，而这些Web作业总共消耗了太多的CPU.但坦率地说，我发现这些症状非常令人误解.

Sooo... I feel a little bit dumb but the issue was not in the code as I surmised. It simply was a problem of CPU usage. I eventually discovered (with the help of Azure support) that I had other web jobs on the same App Service Plan and altogether, these web jobs were consuming too much CPU. But frankly, I find the symptoms very misleading.

无论如何，我希望这会在将来对某人有所帮助.

Anyway, I hope this will help someone in the future.

TL; DR 始终检查您的CPU使用率！

TL;DR Always check your CPU usage!

这篇关于从FTP下载文件仅在Azure Web应用程序中失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！