如何使用.NET将大文件上传到Azure容器? [英] How to upload large files to azure container using .NET?

问题描述

我试图使用Windows窗体应用程序将大文件上传到azure容器.

I was trying to upload large files to an azure container using a windows form application.

由于文件很大，因此我无法将其作为单个块上传.

Since the file size was large, I couldn't upload it as a single block.

找到了一种以块为单位上传大文件的方法. 我将代码发布在这里，希望可以帮助有类似要求的人.

A method to upload large files as set of blocks were figured out. I am posting the code here hoping to help someone having similar requirement.

推荐答案

我们可以使用Block blobs将大型文件上传到Azure容器.

We can upload large files to azure container by using Block blobs.

Block blobs由多个块组成，每个块均由一个块ID标识.

Block blobs are comprised of blocks, each of which is identified by a block ID.

当我们将一个块上传到Blob时，它与指定的块Blob相关联，但是在您提交包含新块ID的块列表之前，它不会成为Blob的一部分.

When we upload a block to a blob, it is associated with the specified block blob, but it does not become part of the blob until you commit a list of blocks that includes the new block's ID.

块ID是blob中长度相等的字符串.

Block IDs are strings of equal length within a blob.

块客户端代码通常使用base-64编码将字符串标准化为相等的长度. 使用base-64编码时，预编码的字符串必须小于等于64个字节.

Block client code usually uses base-64 encoding to normalize strings into equal lengths. When using base-64 encoding, the pre-encoded string must be 64 bytes or less.

有关更多信息，请阅读文档

For more info read the documentation here.

以下代码将源文件分割为多个字节数组，每个字节数组的大小为10MB. 每个字节数组都使用 放入块 操作. 这些块将与指定的Block blob关联.

The following code, splits the source file into multiple byte arrays of size 10MB each. Each byte array is uploaded as blocks using Put Block operation. These blocks will be associated with the specified Block blob.

以后，使用 来提交blockID.放置阻止列表 操作，该操作将使用blockID从上传的块中创建blob.

Later the blockIDs are commited using Put Block List operation, which will create the blob from the uploaded blocks using the blockIDs.

public string UploadFile(string sourceFilePath)
{
    try
    {
        string storageAccountConnectionString = "AZURE_CONNECTION_STRING";
        CloudStorageAccount StorageAccount = CloudStorageAccount.Parse(storageAccountConnectionString);
        CloudBlobClient BlobClient = StorageAccount.CreateCloudBlobClient();
        CloudBlobContainer Container = BlobClient.GetContainerReference("container-name");
        Container.CreateIfNotExists();
        CloudBlockBlob blob = Container.GetBlockBlobReference( Path.GetFileName(sourceFilePath) );
        HashSet<string> blocklist = new HashSet<string>();

        byte[] fileContent = File.ReadAllBytes(sourceFilePath);
        const int pageSizeInBytes = 10485760;
        long prevLastByte = 0;
        long bytesRemain = fileContent.Length;

        do
        {
            long bytesToCopy = Math.Min(bytesRemain, pageSizeInBytes);
            byte[] bytesToSend = new byte[bytesToCopy];
            Array.Copy(fileContent, prevLastByte, bytesToSend, 0, bytesToCopy);
            prevLastByte += bytesToCopy;
            bytesRemain -= bytesToCopy;

            //create blockId
            string blockId = Guid.NewGuid().ToString();
            string base64BlockId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId));

            blob.PutBlock(
                base64BlockId,
                new MemoryStream(bytesToSend, true),
                null
                );

            blocklist.Add(base64BlockId);

        } while (bytesRemain > 0);

        //post blocklist
        blob.PutBlockList(blocklist);

        return "Success";
    }
    catch (Exception ex)
    {
        return ex.Message;
    }
}

这篇关于如何使用.NET将大文件上传到Azure容器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！