删除Azure数据工厂UI中所需的Blob活动 [英] Delete Blob Activity Needed in Azure Data Factory UI
问题描述
Azure门户中的数据工厂UI包含多个活动，可以在创建数据工厂管道时将其链接起来，例如复制活动。我使用此活动从blob中读取CSV文本文件，解析其内容，并将它们
复制到SQL Server数据库。这就像一个魅力。但是，在Copy活动成功完成其工作之后，我想通过将其移动到另一个位置（将其移动到另一个存储容器或
相同容器内的另一个文件夹）来存档此blob。似乎没有可用的活动可以帮助我解决这个问题。
我想建议一个新的"删除Blob"。将从存储容器中删除blob的活动。然后，我可以实现复制到SQL Server数据库然后归档CSV文件的目标。为此，我将链接一个Copy活动（
将数据复制到SQL Server）链接到另一个Copy活动（将blob移动到存档），然后链接到Delete Blob活动（删除包含的blob）已存档）。通过浏览在线论坛，这似乎是Azure
数据工厂的许多用户的基本方案。我当然不是唯一一个寻找从管道中删除blob的方法。
MSDN上的当前文档建议创建一个新的Azure Batch来运行自定义构建的.NET控制台将执行blob删除的应用程序。这看起来有点矫枉过正。为什么我必须编写整个控制台应用程序并为任务创建一个
Azure批处理作为删除blob的平凡事件？特别是，Azure数据工厂UI已包含复制活动时。删除Blob或Move Blob活动看起来就像它们自然适合复制活动一样。当删除blob这样简单的事情不是开箱即用的活动之一时，使用UI设置
管道的承诺感觉很糟糕。
我希望Azure团队考虑将"删除Blob"或"移动Blob"活动添加到数据工厂UI。谢谢！
非常感谢您的反馈和建议！这绝对是我们在ADF中更简单的一项活动。
与此同时，这是一种删除Azure存储Blob和Azure表行的技术，没有自定义活动或Azure批处理，使用REST API：
以下是来自ADF的Web Activity的直接REST API调用，用于删除ADLS中的文件：
https://mssqldude.wordpress.com/2018/03/27 /天青-数据工厂删除-文件从 - 天青数据湖-STORE-ADL的/
The Data Factory UI in Azure Portal includes multiple activities that one can chain up when creating a Data Factory pipeline, like the Copy activity for instance. I use this activity to read a CSV text file from a blob, parse its contents, and copy them to a SQL Server database. This works like a charm. However, after the Copy activity completes its work successfully, I would like to archive this blob by moving it to another location (either to a different storage container or to another folder within the same container). There does not seem to be an activity available that can help me with this.
I would like to suggest a new "Delete Blob" activity that would delete a blob from a storage container. I can then achieve the goal of copying to a SQL Server database and then archive the CSV file. To do this, I would chain one Copy activity (to copy the data to SQL Server) to another Copy activity (to move the blob to the archive) and then to a Delete Blob activity (to remove the blob that has been archived). From looking around online forums, this seems to be a basic scenario for many users of Azure Data Factory. I am certainly not the only one looking for a way to delete a blob from within a pipeline.
Current documentation on MSDN suggests creating a new Azure Batch to run a custom-build .NET Console Application that will perform the deleting of the blob. This seems like overkill. Why should I have to program an entire console application and create an Azure Batch for a task as mundane as deleting a blob? Especially, when the Azure Data Factory UI already includes a Copy activity. The Delete Blob or Move Blob activities seem like they would naturally fit alongside the Copy activity. The promise of setting up pipelines with the UI feels broken when something as simple as deleting a blob is not one of the activities included out of the box.
I hope the Azure team considers adding a Delete Blob or Move Blob activity to the Data Factory UI. Thanks!
Thank you so much for the feedback and suggestions! This is absolutely an activity that we will make simpler in ADF.
In the meantime, here is a technique to delete Azure Storage Blobs and Azure Table rows w/o custom activities or Azure Batch, using the REST API:
And here is a direct REST API call from ADF's Web Activity to delete files in ADLS:
这篇关于删除Azure数据工厂UI中所需的Blob活动的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
