StorageException通过慢速网络下载大文件时 [英] StorageException when downloading a large file over a slow network
问题描述
我使用的NuGet包 WindowsAzure.Storage
版本4.2.1。
I'm using the NuGet package WindowsAzure.Storage
version 4.2.1.
以下code试图从在一个遥远的数据中心存储容器下载Blob的。
The following code tries to download a blob from a storage container that is in a distant datacenter.
try
{
var blobRequestOptions = new BlobRequestOptions
{
RetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(5), 3),
MaximumExecutionTime = TimeSpan.FromMinutes(60),
ServerTimeout = TimeSpan.FromMinutes(60)
};
using (var fileStream = File.Create(localPath))
{
blockBlob.DownloadToStream(fileStream, null, blobRequestOptions);
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
不过,有时它下载了大约10分钟,然后将它抛出以下异常:
However, sometimes it downloads for ~10 minutes and then it throws the following exception:
Unhandled Exception: Microsoft.WindowsAzure.Storage.StorageException: The client could not finish the operation within specified timeout. ---> System.TimeoutException: The client could not finish the operation within specified timeout.
--- End of inner exception stack trace ---
at Microsoft.WindowsAzure.Storage.Core.Util.StorageAsyncResult`1.End()
at Microsoft.WindowsAzure.Storage.Blob.CloudBlockBlob.EndUploadText(IAsyncResult asyncResult)
at Microsoft.WindowsAzure.Storage.Core.Util.AsyncExtensions.<>c__DisplayClass4.b__3(IAsyncResult ar)
--- End of stack trace from previous location where exception was thrown ---
我该如何解决这个问题?
推荐答案
请试试这个code。基本上它是它会创建一个空文件,然后再使用1 MB块读取blob数据<一个href=\"https://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.blob.cloudblockblob.downloadrangetostream.aspx\"相对=nofollow> DownloadRangeToStream
方法。由于大块获取下载,它是附加到文件中。
Please try this code. Basically what it does is it creates an empty file first and then read the blob's data in 1 MB chunks using DownloadRangeToStream
method. As the chunks get downloaded, it appends it to the file.
private static void DownloadLargeFile()
{
var account = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
var blobClient = account.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("container-name");
var file = "my-very-large-file-name";
var blob = container.GetBlockBlobReference(file);
//First fetch the size of the blob. We use this to create an empty file with size = blob's size
blob.FetchAttributes();
var blobSize = blob.Properties.Length;
long blockSize = (1 * 1024 * 1024);//1 MB chunk;
blockSize = Math.Min(blobSize, blockSize);
//Create an empty file of blob size
using (FileStream fs = new FileStream(file, FileMode.Create))//Create empty file.
{
fs.SetLength(blobSize);//Set its size
}
var blobRequestOptions = new BlobRequestOptions
{
RetryPolicy = new ExponentialRetry(TimeSpan.FromSeconds(5), 3),
MaximumExecutionTime = TimeSpan.FromMinutes(60),
ServerTimeout = TimeSpan.FromMinutes(60)
};
long currentPointer = 0;
long bytesRemaining = blobSize;
do
{
var bytesToFetch = Math.Min(blockSize, bytesRemaining);
using (MemoryStream ms = new MemoryStream())
{
//Download range (by default 1 MB)
blob.DownloadRangeToStream(ms, currentPointer, bytesToFetch, null, blobRequestOptions);
ms.Position = 0;
var contents = ms.ToArray();
using (var fs = new FileStream(file, FileMode.Open))//Open that file
{
fs.Position = currentPointer;//Move the cursor to the end of file.
fs.Write(contents, 0, contents.Length);//Write the contents to the end of file.
}
currentPointer += contents.Length;//Update pointer
bytesRemaining -= contents.Length;//Update bytes to fetch
}
}
while (bytesRemaining > 0);
}
我试过在一个小本文件code(差互联网连接:P)。与500 MB的文件尝试之前 - 所以,请您先尝试一下在一个小文件(10 MB左右说5)。 HTH。
I've tried this code on a small file (Poor Internet Connectivity :P). So please try it out first on a small file (say around 5 - 10 MB) before trying with your 500 MB file. HTH.
这篇关于StorageException通过慢速网络下载大文件时的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!