Azure数据工厂:设置限制以使用“复制"活动复制文件数 [英] Azure Data Factory : Set a limit to copy number of files using Copy activity

问题描述

我的管道中有一个复制活动，用于从第2代Azure数据中复制文件.源位置可能有1000个文件，并且文件需要复制，但是我们需要设置所需文件数的限制被复制.ADF中是否有任何选项可以实现相同的限制自定义活动?

I have a copy activity used in my pipeline to copy files from Azure data Lake gen 2. The source location may have 1000's of files and the files are required to be copied but we need to set a limit for the number files required to be copied. Is there any option available in ADF to achieve the same barring a custom activity?

例如:我在Data Lake中有2000个文件，但是在运行管道时，我应该能够传递一个参数以仅复制500个文件.

Eg: I have 2000 files available in Data lake, but while running the pipeline i should able to pass a parameter to copy only 500 files.

关于，桑迪普

推荐答案

我认为您可以将lookup活动与for每个循环和copy活动一起使用以实现此目的.您还必须使用一个计数器变量(这会使过程变慢，因为您必须一次复制i文件).Loopkup活动的当前限制为5000，因此您必须牢记这一点.

I think you can use a lookup activity with the for each loop and a copy activity to achieve this . You will have to use a counter variable also ( this is make the process slow , as you will have to copy i file at a time ) . The loopkup acitivty has a limit of 5000 at this time , so you will have to keep that in mind .

这篇关于Azure数据工厂:设置限制以使用“复制"活动复制文件数的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！