我无法使用单个数据工厂管道加载批量数据，SOURCE = Blob / Redshift，DESTINATION = Azure SQL数据仓库 [英] I am not able to load bulk data using single pipeline of data factory, SOURCE=Blob/Redshift, DESTINATION=Azure SQL Data Warehouse
问题描述
我已按照Microsoft文档执行此操作但失败了。
I have followed Microsoft document to do so but got failed.
请建议或指导其他一些方法。
Please suggest or guide some other way to do so.
迭代和ForEach运行不正常。
Iteration and ForEach is not working well.
否则，我想将Blob Storage中的多个表移动到数据仓库。 （而我使用自托管IR连接到存储帐户）。
Else, I want move multiple table from Blob Storage to data Warehouse. (whereas, I am using Self-Hosted IR to connect to Storage Account).
1。速度非常慢。
2。如果数据很大，则要求实施分段，但用作Blob存储的分段失败。 （数据上传到阶段已通过，但从阶段到仓库在没有启动的情况下失败）
2. If data is huge, it ask for staging to be implemented, but staging used as Blob storage got failed. (Data upload to stage passed, but from stage to warehouse got failed without start)
3。如果我们使用通配符选择多个文件，它会失败并且列与所需的列不匹配（如果我们预览数据，则跳过最后一列，因为它是空白的）
3. If we select multiple files by using wildcard, it gets failed and column doesn't match with required (if we preview data, last column was skipped as it was blank)
推荐答案
你好Gaurav，
Hello Gaurav ,
我们看到你提到的这里有多个问题。让我们尝试一次通过一个。
We see that you are mentioning multiple issues here . Lets try to talk through one at a time .
- ADF很慢。
- ADF is slow .
当你试图将数据从blob复制到SQLDWH你使用了"复制并行度"吗？ ？我建议您也可以查看监控 - >公制并查看数据点以及是否出现问题。
When you tried to copy data from blob to SQLDWH did you used the "degree of copy parallelism" ? I suggest you can also look into the Monitoring -> metric and look at the data points and if something is wrong there .
我们不确定您使用的是哪种IR，但请查看IR日志并查看资源不低。
We are not sure as to what kind of IR you guys are using , but please look at the IR logs and see it is not low on resources .
-errors
-errors
您还提到了很多错误，请分享是错误，以便我们可以进一步研究。
You have also mentioned about lot of errors , please do share with is the errors so that we can look into this further .
这篇关于我无法使用单个数据工厂管道加载批量数据，SOURCE = Blob / Redshift，DESTINATION = Azure SQL数据仓库的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
