Azure数据工厂-基于事件的多个文件/blob触发器 [英] Azure Data Factory - Event based triggers on multiple files/blobs

问题描述

当在Blob容器内的文件夹中创建新文件/blob时，我通过基于事件的触发器调用ADF V2管道.

I am invoking an ADF V2 pipeline via an event based trigger when new files/blobs are created in a folder within a blob container.

Blob容器结构:

BlobContainer-> FolderName->

-> File1.csv

-> File2.csv

-> File3.csv

我使用以下配置创建了触发器:

I've created the trigger with below configuration:

容器名称: BlobContainer

Blob路径以: FolderName/

Blob路径以 .csv

已检查事件:Blob已创建

Event Checked:Blob Created

触发屏幕截图

问题:临时在文件夹中创建了三个csv文件.调用管道的触发器运行3次(可能是因为创建了3个Blob).管道实际上将文件移动到另一个Blob容器中.因此，第一个触发器运行成功，而第2个触发器运行失败，因为文件已被移动.但是我如何配置触发器，以便即使在其中创建了3个文件，每个文件夹也只能运行一次?

Problem: Three csv files are created in the folder on ad hoc basis. The trigger that invokes the pipeline runs 3 times (probably because 3 blobs are created). The pipeline actually move the files in another blob container. So the 1st trigger run succeeds and remaining 2 fails because the files have been moved already. However how can I configure the trigger so that it only run once per folder even though 3 files are created within it?

由于文件是一起生成的，因此我需要使用ADF将它们一起移动到新位置.

Because the files are generated together, I am required to move them together into a new location using ADF.

推荐答案

您的blobEventTrigger触发了每个文件的管道，为此，您可以使用查找活动"来获取文件名，然后使用过滤活动来过滤必需的文件名，并提供可以在IF活动中检查的filterdItemCounts属性.没有文件时，filterdItemCounts返回"0"，并且不会触发您的管道.

Your blobEventTrigger triggered the pipeline for each file, For it, you can use a 'lookup activity' which gets the filenames and then use filter activity, which filtered the required filename and gives the filterdItemCounts attribute that could be checked in the IF Activity. When there is no file the filterdItemCounts returns '0'and your pipeline not triggered.

摘要-查找活动->过滤器活动->中频活动->您的管道

Summary- Lookup Activity -> Filter Activity -> IF Activity -> Your Pipeline

这篇关于Azure数据工厂-基于事件的多个文件/blob触发器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！