我知道要分块上传，我们是否必须在接收端做些事情？ [英] I know about uploading in chunks, do we have to do something on receiving end?

问题描述

我的azure函数接收大型视频文件和图像，并将其存储在Azure blob中。客户端API将数据块发送到我的Azure htttp触发函数。我必须在接收端做一些事情来提高性能，例如接收大块数据吗？

my azure function receives large video files and images and stores it in Azure blob. Client API is sending data in chunks to my Azure htttp trigger function. Do I have to do something at receiving-end to improve performance like receiving data in chunks?

Bruce，实际上客户代码是由其他团队开发的。现在，我正在通过邮递员进行测试，并从多部分HTTP请求中获取文件。

Bruce, actually Client code is being developed by some other team. right now i am testing it by postman and getting files from multipart http request.

foreach (HttpContent ctnt in provider.Contents)  {

                var dataStream = await ctnt.ReadAsStreamAsync();
 if (ctnt.Headers.ContentDisposition.Name.Trim().Replace("\"", "") == "file")
               {                
                        byte[] ImageBytes = ReadFully(dataStream);
                        var fileName = WebUtility.UrlDecode(ctnt.Headers.ContentDisposition.FileName);                         

              } }

ReadFully函数

ReadFully Function

 public static byte[] ReadFully(Stream input){
using (MemoryStream ms = new MemoryStream())
{
    input.CopyTo(ms);
    return ms.ToArray();
}}

推荐答案

为 BlobRequestOptions.ParallelOperationThread 状态如下：

获取或设置可能的块数

Gets or sets the number of blocks that may be simultaneously uploaded.

备注：

在a上使用UploadFrom *方法时blob，该blob将被分解为多个块。设置此
值将限制库在给定时间具有运行中的
的未完成I / O放置块请求的数量。默认值为1（无并行性）。将此值设置得较高可能会导致
更快的blob上传，具体取决于客户端和Azure存储服务之间的网络。
如果blob小（小于256 MB），建议将此值保持为1。

When using the UploadFrom* methods on a blob, the blob will be broken up into blocks. Setting this value limits the number of outstanding I/O "put block" requests that the library will have in-flight at a given time. Default is 1 (no parallelism). Setting this value higher may result in faster blob uploads, depending on the network between the client and the Azure Storage service. If blobs are small (less than 256 MB), keeping this value equal to 1 is advised.

我认为您可以显式设置 ParallelOperationThreadCount 以便更快地上传Blob。

I assumed that you could explicitly set the ParallelOperationThreadCount for faster blob uploading.

var requestOption = new BlobRequestOptions()
{
    ParallelOperationThreadCount = 5 //Gets or sets the number of blocks that may be simultaneously uploaded.
};

//upload a blob from the local file system
await blockBlob.UploadFromFileAsync("{your-file-path}",null,requestOption,null);

//upload a blob from the stream
await blockBlob.UploadFromStreamAsync({stream-for-upload},null,requestOption,null);

foreach（provider.Contents中的HttpContent ctnt）

foreach (HttpContent ctnt in provider.Contents)

根据您的代码，我假设您按以下方式检索 provider 实例：

Based on your code, I assumed that you retrieve the provider instance as follows:

MultipartMemoryStreamProvider provider = await request.Content.ReadAsMultipartAsync();

此时，您可以使用以下代码上传新的Blob：

At this time, you could use the following code for uploading your new blob:

var blobname = ctnt.Headers.ContentDisposition.FileName.Trim('"');
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobname);
//set the content-type for the current blob
blockBlob.Properties.ContentType = ctnt.Headers.ContentType.MediaType;
await blockBlob.UploadFromStreamAsync(await ctnt.Content.ReadAsStreamAsync(), null,requestOption,null);

我宁愿使用 MultipartFormDataStreamProvider 将从中存储上传的文件客户端到文件系统，而不是 MultipartMemoryStreamProvider ，它将使用服务器内存临时存储从客户端发送的数据。方法，您可以遵循类似的问题。此外，我希望将Azure存储客户端库与我的Azure函数一起使用，可以遵循使用.NET入门Azure Blob存储。

I would prefer use MultipartFormDataStreamProvider which would store the uploaded files from the client to the file system instead of MultipartMemoryStreamProvider which would use the server memory for temporarily storing the data sent from the client. For the MultipartFormDataStreamProvider approach, you could follow this similar issue. Moreover, I would prefer use the Azure Storage Client Library with my Azure function, you could follow Get started with Azure Blob storage using .NET.

更新： >

此外，您可以按照以下教程，内容涉及将大文件分成小块，在客户端上传，然后在服务器端合并。

Moreover, you could follow this tutorial about breaking a large file into small chunks, upload them in the client side, then merge them back in your server side.

这篇关于我知道要分块上传，我们是否必须在接收端做些事情？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！