复制和移动数据时进行文件验证并重新发送 [英] File validation and re-send when copying and moving data

问题描述

您好，Azure专家

我有一个潜在的客户，希望使用ADF V2进行数据集成.

I have a potential customer who wishes to use ADF V2 for data integration.

他们希望将超大文件从本地服务器移动并复制到云服务器，然后可能使用ADF进行数据集成.

They want to move and copy very large files over from on-premise server to cloud server and probably use ADF to do the data integration after that.

我阅读了这篇文章-https://docs.microsoft.com/zh-cn/azure/data-factory/connector-file-system

I read this article - https://docs.microsoft.com/en-us/azure/data-factory/connector-file-system

我有以下问题:-

1. ADF复制功能可以对源文件和复制文件执行文件验证(MD5，CRC等)吗?

1. Can ADF copy function perform file validation (MD5, CRC, etc.) of the source and copied files?

2.如果在文件移动/复制期间网络断开连接，ADF复制功能可以重新发送文件吗?

2. In case the network disconnects during file move/copy, can ADF copy function re-send the file?

非常感谢您.

最诚挚的问候.

推荐答案

你好

1.是的，获取元数据"活动支持MD5:

1.  Yes, the Get Metadata activity supports MD5:

https://docs.microsoft.com/zh-cn/azure/data-factory/control-flow-get-metadata-activity

https://docs.microsoft.com/en-us/azure/data-factory/control-flow-get-metadata-activity

2.是的，管道活动支持可配置的重试尝试:

2.  Yes, pipeline activities support configurable retry attempts:

https://docs.microsoft.com/zh-cn/azure/data-factory/concepts-pipelines-activities#activity-policy

https://docs.microsoft.com/en-us/azure/data-factory/concepts-pipelines-activities#activity-policy

我建议您看一下有关复制活动"性能调整的文档.您可能会发现测试并行复制，分段复制和横向扩展IR等功能很有帮助:

I would recommend taking a look at the documentation on Copy Activity performance tuning.  You may find it beneficial to test features such as Parallel Copy, Staged Copy, and scale out IR:

https://docs.microsoft.com/zh-CN/azure/data-factory/copy-activity-performance

https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-performance

我希望这会有所帮助！ :)

I hope this helps! :)

这篇关于复制和移动数据时进行文件验证并重新发送的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！