Azure DataFactory增量BLOB副本 [英] Azure DataFactory Incremental BLOB copy

问题描述

我已经建立了将数据从一个Blob存储复制到另一个Blob的管道.我希望有可能的增量副本，但是还没有找到指定它的方法.原因是我想按计划运行它，并且仅复制自上次运行以来的任何新数据.

I've made a pipeline to copy data from one blob storage to another. I want to have incremental copy if it's possible, but haven't found a way to specify it. The reason is I want to run this on a schedule and only copy any new data since last run.

推荐答案

1. 如果您的Blob名称用时间戳记很好地命名，则可以按照此文档操作复制分区数据.您可以使用获取元数据活动以检查上次修改时间.请参考这篇文章.
>

1. If your blob name is well named with timestamp, you could follow this doc to copy partitioned data. You could use copy data tool to setup the pipeline. You could select tumbling window and then in file path filed input {year}/{month}/{day}/fileName and choose the right pattern. It will help you construct the parameters.
2. If you blob name is not well named with timestamp, you could use get metadata activity to check the last modified time. Please reference this post.

事件触发器只是控制管道何时运行的一种方法.您还可以使用滚动窗口触发器或计划触发器在您的方案中.

Event trigger is just one way to control when the pipeline should run. You could also use tumbling window trigger or schedule trigger in your scenarios.

这篇关于Azure DataFactory增量BLOB副本的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！