如何从一个网页上传大文件到Azure的BLOB [英] How to upload huge files to the Azure blob from a web page
问题描述
在内部ASP.NET有2 GB的寻址空间，但在现实中你只有不到1 GB免费上传（请参见 http://support.microsoft.com/?id=295626 ）。此外IIS 7拥有30 MB的上限（请参阅http://www.iislogs.com/steveschofield/iis7-post-40-adjusting-file-upload-size-in-iis7 ），你理应必须运行
Internally ASP.NET has a 2 GB addressing space, but in reality you only have less than 1 GB free for uploads (see http://support.microsoft.com/?id=295626 ). In addition IIS 7 has a cap of 30 MB ( see http://www.iislogs.com/steveschofield/iis7-post-40-adjusting-file-upload-size-in-iis7 ) and you supposedly have to run
appcmd set config "My Site/MyApp" -section:requestFiltering -requestLimits.maxAllowedContentLength:104857600 -commitpath:apphost
在服务器上超越这个30 MB的限制。但是，我怎么能在我的Azure服务器上运行呢？
on the server to go beyond this 30 MB limit. But how can I run this on my Azure servers?
此外，根据 http://support.microsoft.com/?id=295626
在上载过程中，ASP.NET
加载到内存之前，整个文件
用户可以将文件保存到
磁盘。
During the upload process, ASP.NET loads the whole file in memory before the user can save the file to the disk.
，所以我会很快，如果很多用户同时上传大文件耗尽内存限制。在我的code下面我使用流，但我猜，整个文件在内存中上传第一反正。这样的话？
, so I will quickly exhaust the memory limit if many users upload large files at one time. In my code below I use streams, but I'm guessing that the whole file is uploaded in memory first anyway. Is this the case?
using System;
using System.Web.Security;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;
namespace WebPages
{
public partial class Upload : System.Web.UI.Page
{
CloudBlobClient BlobClient = null;
CloudBlobContainer BlobContainer = null;
void InitBlob()
{
// Setup the connection to Windows Azure Storage
var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
BlobClient = storageAccount.CreateCloudBlobClient();
// Get and create the container
BlobContainer = BlobClient.GetContainerReference("publicfiles");
}
protected void Page_Load(object sender, EventArgs e)
{
//if (Membership.GetUser() == null) return; // Only allow registered users to upload files
InitBlob();
try
{
var file = Request.Files["Filedata"];
var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
BlobClient = storageAccount.CreateCloudBlobClient();
// Make a unique blob name
var extension = System.IO.Path.GetExtension(file.FileName);
// Create the Blob and upload the file
var blobAddressUri = String.Format("{0}{1}", Guid.NewGuid(), extension);
var blob = BlobContainer.GetBlobReference(blobAddressUri);
blob.UploadFromStream(file.InputStream);
// Set the metadata into the blob
blob.Metadata["FileName"] = file.FileName;
//blob.Metadata["Submitter"] = Membership.GetUser().UserName;
blob.Metadata["Type"] = "Video";
blob.Metadata["Description"] = "Test";
blob.SetMetadata();
// Set the properties
blob.Properties.ContentType = file.ContentType;
blob.SetProperties();
}
catch(Exception ex)
{
System.Diagnostics.Trace.TraceError("Upload file exception: {0}", ex.ToString());
// If any kind of error occurs return a 500 Internal Server error
Response.StatusCode = 500;
Response.Write("An error occured while uploading the file");
Response.End();
}
}
}
}
我知道非网页上传工具，如 HTTP：//azureblobuploader.$c$cplex.com/ ，但我真的需要它从网页上载。
I am aware of non web page upload tools like http://azureblobuploader.codeplex.com/ , but I really need it to be uploaded from a web page.
所以，我的问题是：
- 如何我上传的文件是大于2 GB从网页团块
- 如何从一个网页作为流我大的上传文件，不吃掉所有的内存
- 如果解决的办法就是写我自己的HttpModule或HttpHandler的处理我上传，我怎么能得到这个安装在我的Azure服务器？我可以使用HttpHandlers的如 HTTP：在Azure上//neatupload.$c$cplex.com/
- 这个项目是不是在SharePoint上，但我knwo在SharePoint，你有一些所谓的Blob提供商和您可以编写自己的，是否有斑点ASP.NET提供？
我也可以提，我上面的code正常工作在默认情况下超过30 MB的小文件，我用的SWFUpload V2.2.0客户端上。
I can also mention that my code above works fine by default with files smaller than 30 MB, I use SWFUpload V2.2.0 on the client.
更新19月19:09：
@YvesGoeleven在Twitter上给我使用共享访问签名（见msdn.microsoft.com/en-us/library/ee395415.aspx），并直接上传文件到的Azure Blob存储的尖端不通过ASP.NET要在所有。我创建了一个JSON WCF返回一个有效的SAS UT我Blob存储。
Update 19. June 19:09: @YvesGoeleven on Twitter gave me a tip of using Shared Access Signature (see msdn.microsoft.com/en-us/library/ee395415.aspx ) and uploading the file directly to the Azure Blob Storage without going through the ASP.NET at all. I created a JSON WCF that returns a valid SAS ut to my blob storage.
using System.ServiceModel;
using System.ServiceModel.Web;
namespace WebPages.Interfaces
{
[ServiceContract]
public interface IUpload
{
[OperationContract]
[WebInvoke(Method = "GET",
ResponseFormat = WebMessageFormat.Json)]
string GetUploadUrl();
}
}
--------
using System;
using System.IO;
using System.Runtime.Serialization.Json;
using System.ServiceModel.Activation;
using System.Text;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;
namespace WebPages.Interfaces
{
[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
public class UploadService : IUpload
{
CloudBlobClient BlobClient;
CloudBlobContainer BlobContainer;
public UploadService()
{
// Setup the connection to Windows Azure Storage
var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
BlobClient = storageAccount.CreateCloudBlobClient();
// Get and create the container
BlobContainer = BlobClient.GetContainerReference("publicfiles");
}
string JsonSerialize(string url)
{
var serializer = new DataContractJsonSerializer(url.GetType());
var memoryStream = new MemoryStream();
serializer.WriteObject(memoryStream, url);
return Encoding.Default.GetString(memoryStream.ToArray());
}
public string GetUploadUrl()
{
var sasWithIdentifier = BlobContainer.GetSharedAccessSignature(new SharedAccessPolicy()
{
Permissions = SharedAccessPermissions.Write,
SharedAccessExpiryTime =
DateTime.UtcNow.AddMinutes(60)
});
return JsonSerialize(BlobContainer.Uri.AbsoluteUri + "/" + Guid.NewGuid() + sasWithIdentifier);
}
}
}
它的工作原理，但由于它使用了HTTP POST动词，而不是HTTP PUT动词在Azure Blob存储创建一个新的BLOB项目时预计，我不能SWFUpload的使用它。任何人都知道如何解决这个问题未做使用HTTP PUT动词自己的自定义的Silverlight或Flash客户端组件？在上传文件时，因此，使用把提交的表单是不是最优的，我想一个进度条。
It works, but I can't use it with SWFUpload since it uses the HTTP POST verb and not the HTTP PUT verb that the Azure Blob Storage expects when creating a new blob item. Anyone know how to get around this without making my own custom Silverlight or Flash client component that uses the HTTP PUT verb? I wanted a progress bar when uploading the files, therefore a submitted form that uses PUT is not optimal.
对于那些有兴趣在客户端code（因为SWFUpload的是不会工作使用HTTP POST并没有投入作为Azure的Blob存储预计）：
For those interested in the client code (that wont work since SWFUpload uses HTTP POST and not PUT as Azure Blob Storage expects):
<div id="header">
<h1 id="logo"><a href="/">SWFUpload</a></h1>
<div id="version">v2.2.0</div>
</div>
<div id="content">
<h2>Application Demo (ASP.Net 2.0)</h2>
<div id="swfu_container" style="margin: 0px 10px;">
<div>
<span id="spanButtonPlaceholder"></span>
</div>
<div id="divFileProgressContainer" style="height: 75px;"></div>
<div id="thumbnails"></div>
</div>
</div>
<script type="text/javascript" language="javascript">
$(document).ready(function () {
$.ajax({
url: '/Interfaces/UploadService.svc/GetUploadUrl',
success: function (result) {
var parsedResult = $.parseJSON(result);
InitUploadFile(parsedResult);
}
});
function InitUploadFile(uploadUrl) {
//alert(uploadUrl);
var swfu = new SWFUpload({
// Backend Settings
upload_url: uploadUrl,
post_params: {
"ASPSESSID": "<%=Session.SessionID %>"
},
// File Upload Settings
file_size_limit: "100 MB",
file_types: "*.*",
file_types_description: "All file types",
file_upload_limit: "0", // Zero means unlimited
// Event Handler Settings - these functions as defined in Handlers.js
// The handlers are not part of SWFUpload but are part of my website and control how
// my website reacts to the SWFUpload events.
file_queue_error_handler: fileQueueError,
file_dialog_complete_handler: fileDialogComplete,
upload_progress_handler: uploadProgress,
upload_error_handler: uploadError,
upload_success_handler: uploadSuccess,
upload_complete_handler: uploadComplete,
// Button settings
button_image_url: "Images/swfupload/XPButtonNoText_160x22.png",
button_placeholder_id: "spanButtonPlaceholder",
button_width: 160,
button_height: 22,
button_text: '<span class="button">Select files <span class="buttonSmall">(2 MB Max)</span></span>',
button_text_style: '.button { font-family: Helvetica, Arial, sans-serif; font-size: 14pt; } .buttonSmall { font-size: 10pt; }',
button_text_top_padding: 1,
button_text_left_padding: 5,
// Flash Settings
flash_url: "Js/swfupload-2.2.0/swfupload.swf", // Relative to this file
custom_settings: {
upload_target: "divFileProgressContainer"
},
// Debug Settings
debug: false
});
}
});
</script>
更新19月21:07：
我想既然SWFUpload的是，我下载源代码并改变从POST动词把开源的，可悲的是Flash Player的URLRequestMethod不支持其他动词不是GET和POST。我没有找到一个所谓的变通
I figured since SWFUpload is open source that I download the source and change the verb from POST to PUT, sadly the Flash Player URLRequestMethod does not support other verbs than GET and POST. I did find a supposed work-around
private function BuildRequest():URLRequest {
// Create the request object
var request:URLRequest = new URLRequest();
request.method = URLRequestMethod.POST;
request.requestHeaders.push(new URLRequestHeader("X-HTTP-Method-Override", "PUT"));
，但在Adobe AIR和不使用Flash Player这只是工作。
, but this only work in Adobe Air and not with the Flash Player.
我读过的Silverlight 3中，后来支持HTTP PUT动词，所以我觉得我必须写一些SilverLight的code到这里得到我的路。我发现这个博客系列文章，这将可能帮助我在这里<一个href=\"http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures\">http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures
I've read that SilverLight 3 and later supports the HTTP PUT verb, so I think I have to write some SilverLight code to get my way here. I did find this blog article series that will probably help me here http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures .
更新@ 6月27日'11：
我现在已经成功地管理使用自定义Silverlight客户端根据<一个项目我写上传从网页大文件（4,5 GB的文件进行测试） href=\"http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures\">http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures 。由于Silverlight支持这两个的Azure Blob存储要求和支持逐行上传的HTTP PUT动词，我现在已经直接上传大量的文件，以在Azure Blob存储的可能性，我没有去throgh一个ASP.NET的解决方案，我也得到一些不错的进度条，如果他/她想要的用户可以在上传的中间取消。在服务器上的内存使用量是最小的，因为它被放置在在Azure Blob存储之前，把整个文件没有上传。我使用的是共享访问签名（见msdn.microsoft.com/en-us/library/ee395415.aspx）是从一个WCF REST风格的服务提供上的要求。我认为，这个解决方案是我们找到的最好的之一。谢谢你。
I now have successfully managed to upload large files (tested with 4,5 Gb files) from a web page using a custom Silverlight client I wrote based on the project in http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-1-shared-access-signatures . Since Silverlight supports both the HTTP PUT verb that Azure Blob Storage requires and supports progressive uploads, I now have the possibility to upload massive files directly to the Azure Blob Storage and I don't have to go throgh a ASP.NET solution, I also get some nice progress bars and the user can cancel in the middle of the upload if he/she wants to. The memory usage on the server is minimal since the whole file is not uploaded before it is placed in the Azure Blob Storage. I use a Shared Access Signature (see msdn.microsoft.com/en-us/library/ee395415.aspx ) that is supplied from a WCF RESTfull service on request. I think that this solution is the best one we found. Thanks.
更新@ 18月'11：
我已经创建了一个开源项目，我发现在这里：
I have created an open source project with what I found here:
HTTP：//azureslfileuploader.$c$cplex.com/
推荐答案
我实际做同样的事情最近。我创建一个Silverlight客户端应用程序来处理拿刀砍数据并将其发送到Azure上。
I actually did the exact same thing recently. I created a Silverlight Client app to handle chopping up the data and sending it to Azure.
这是我遵循的正是这么做的工作示例。 pretty多关注该和你的工作几乎是太多为你做。
This is a working example that I followed that does exactly that. Pretty much follow this and you're work is almost much done for you.
这篇关于如何从一个网页上传大文件到Azure的BLOB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
