为什么 Azure 数据工厂使用 AppendFile 而不是 PutBlob 将文件写入 Blob 存储容器? [英] Why does Azure Data Factory use AppendFile instead of PutBlob to write files to blob storage container?

问题描述

我有一个 Log Analytics 工作区，并为某些 Azure 存储帐户创建了一些(blob)诊断设置.现在我正在分析 blob 流量.

似乎将 blob 放入 blob 存储的各种方法(Azure 数据工厂 (ADF)、Azure 存储资源管理器 (ASE)、Python SDK 等)使用不同的开箱即用 API 方法.

示例:

• 我看到所有使用 ADF 上传的文件都使用 AppendFile OperationName

• 而使用 ASE 上传的所有文件都使用 PutBlob OperationName

问题:

• AppendFile 是在 ADF 端显式设置的方法吗?
• 还是将 Blob 从 ADF 写入 Blob 存储时的默认设置?

此外，我没有看到 AppendFile 在

当您使用

它与 REST API 示例相同，因为它本质上调用了这个 API.

即使是 Data Lake Storage Gen2 帐户，普通的 Storage REST API 也适用于它，因此如果您使用 Azure Storage Explorer 之类的东西，它本质上是直接调用普通的 Storage REST API，即 Put blob.

I've got a Log Analytics Workspace stood up and created some (blob) Diagnostic Settings for some Azure Storage Accounts. Now I'm analyzing the blob traffic.

Seems that the various methods of getting blobs into blob storage (Azure Data Factory (ADF), Azure Storage Explorer (ASE), Python SDK, etc.) use different API methods out-of-the-box.

Example:

• I'm seeing that all files uploaded using ADF are using the AppendFile OperationName

• Whereas all files uploaded using ASE use the PutBlob OperationName

Question:

• Is AppendFile a method that was explicitly set on the ADF side?
• Or is it the default when writing blobs from ADF to blob storage?

Also, I don't see AppendFile listed as a method in the Blob Service REST API docs.

• Is this just an ADF thing or is the method missing from the docs?

解决方案

I can reproduce your issue on my side, I suppose your storage account is a Data Lake Storage Gen2 account i.e. Hierarchical namespace was enabled like below.

When you use the copy activity in ADF to copy blobs between containers(also named filesystem in datalake gen2), it will call the Data Lake Storage Gen2 REST API instead of the normal Storage REST API - Path - Update, if you look into the Uri parameter in the log, you will find its format is like below.

It is the same as the REST API sample, because it essentially calls this API.

Even if it is a Data Lake Storage Gen2 account, the normal Storage REST API also works for it, so if you use something like Azure Storage Explorer, it essentially calls the normal Storage REST API directly i.e. Put blob.

这篇关于为什么 Azure 数据工厂使用 AppendFile 而不是 PutBlob 将文件写入 Blob 存储容器?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！