Azure ADF V2 ForEach文件从Blob存储复制到SQL表 [英] Azure ADF V2 ForEach File CopyData from Blob Storage to SQL Table

问题描述

我需要设计一个ADF管道，以将在名为当前"的特定Blob存储文件夹路径上创建的CSV文件复制到SQL表.成功复制后，我将不得不将文件移动到存档文件夹.

I need to design an ADF pipeline to copy a CSV file created on a particular Blob Store folder path named "Current" to a SQL table. After successful copy, i'll have to move the file to an archive folder.

我已经完成的事情:我正在使用复制数据活动"，该活动可以复制CSV文件并将其加载到我的SQL表中.成功后，另一个复制数据活动会将CSV文件从当前"文件夹复制到存档"，此后，我将执行删除活动，将其从当前文件夹中删除.

Things i've accomplished: I'm using a Copy Data Activity that copies the CSV file and loads into my SQL table. On success, another copy data activity will copy the CSV file from "Current" folder to "Archive", after this i have a Delete activity that deletes the file from Current Folder.

问题:如果一次只有一个文件，这是完全可以的.但我想重新设计以处理多个文件导入.我想利用ForEach活动.请帮忙.

Issue: This is totally fine where there is only one file at a time. But i want redesign this to handle multiple files imports. I want make use of ForEach activity. Please help.

推荐答案

您需要在for-each之前添加获取元数据"活动.获取元数据"活动将获取当前目录中的文件，并将它们传递给For-Each.您将其连接到Blob存储文件夹并指定文件掩码.活动的输出将需要作为for-each循环的输入.诸如此类.. @ activity('Get Metadata').output.顺便问一下，您如何触发该ADF管道?完成此操作的另一种方法是，设置一个事件触发器以在该Blob目录中创建文件时运行该管道.以下是在存储容器中的文件上使用For-Each的步骤.

You need to add a Get Metadata activity before the for-each. The Get Metadata activity will get the files in the current directory and pass them to the For-Each. You connect it to your Blob storage folder and specify the file mask. The output of the activity will need to be the input for your for-each loop. Something like this..@activity('Get Metadata').output. By the way how are you triggering this ADF pipeline? Another way to accomplish this would be to setup an event trigger to run this pipeline whenever a file is created in that blob directory. Here are the steps to use the For-Each on files in a storage container.

1. 将获取元数据"参数设置为子项"
2. 在您的每次访问中，将项目设置为@activity('Get Metadata1').output.childitems

1. 在复制活动"中使用的源数据集"中，创建一个名为FileName的参数.

1. 设置文件路径以使用参数

1. Set the file path to use the parameter

在复制活动"上，将FileName参数设置为@item().name

On the Copy Activity set the FileName parameter to @item().name

要完成此操作，您需要将接收器设置为所需的任何值.那应该做.

To finish you'll need to set the sink to whatever you need. That should do it.

这篇关于Azure ADF V2 ForEach文件从Blob存储复制到SQL表的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！