Azure数据工厂从查询中获取“针对每个"组件的数据 [英] Azure Data Factory get data for "For Each"component from query
问题描述
情况如下:我的数据库中有一个表，每天接收约300万行.我们希望定期对此表进行存档，以便表中只有最近的8周.其余数据可以存档到AZure Data湖中. 我已经知道一次一次做到这一点的方法.但是现在我想在表的前7天每周运行一次此管道.我假设我应该使用对于每个"组件来执行此操作.它应该沿我要备份的数据集中存在的七个不同日期发出信号.此数据集已从源表直接复制到存档表. 使用SQL查询获取不同的日期并不难，但是如何将查询的结果获取到用于"For Each"组件的数组中?
The situation is as follows: I have a table in my database that recieves about 3 million rows each day. We want to archive this table on a regular base, so that only the 8 most recents weeks are in the table. The rest of the data can be archived tot AZure Data lake. I allready found out how to do this by one day at a time. But now I want to run this pipeline each week for the first seven days in the table. I assume I should do this with the "For Each" component. It should itterate along the seven distinct dates that are present in the dataset I want to backup. This dataset is copied from the source table to an archive table on forehand. It's not difficult to get the distinct dates with a SQL query, but how to get the result of this query into an array that is used for the "For Each" component?
推荐答案
多亏了一位同事，问题得以解决. 我们要做的是为接收器的数据集分配一个参数.命名方式无关紧要，也不必为其分配值.但让我们假设此参数称为日期" 之后，您可以通过使用"@dataset().Date"在接收器的文件名(也在数据集中)中使用此参数. 之后，返回到复制活动，并在接收器中将数据集属性分配给@item().DateSelect. (DateSelect是传递给For Each活动的数组中的字段名称)
The issue is solved thanks to a co-worker. What we have to do is assign a parameter to the dataset of the sink. Does not matter how you name this and you do not have to assign a value to it. But let's assume this parameter is called "date" After that you can use this parameter in the filename of the sink (also in dataset) with by using "@dataset().Date". After that you go back to the copyactivity and in the sink you assign a dataset property to @item().DateSelect. (DateSelect is the field name from the array that is passed to the For Each activity)
另请参阅博希亚的答案作为答案的一部分
See also the answer from Bo Xioa as part of the answer
这样，它可以完美工作.可惜的是，没有得到很好的记录
This way it works perfectly. It's just a shame that this is not well documented
这篇关于Azure数据工厂从查询中获取“针对每个"组件的数据的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
