Azure数据工厂复制活动的性能问题！ [英] Performance Issue with Azure Data Factory copy Activity !!

问题描述

在从Blob存储到SQL DW使用BulkCopy时，我遇到性能问题，因为进行了几次故障排除，所以我一直在进行故障排除 天，但到目前为止没有结果.但是，当我在复制数据"活动中使用Polybase时，它的速度非常快，只需几秒/分钟即可完成，而不是数小时.我正在使用以下高配置，仍然没有帮助. Azure SQL DW – DWU – 1500复制活动–数据 集成单元– 16复制活动–复制程度并行– 20当我使用Polybase时，找不到在某些位置记录被拒绝的行的方法.您能帮忙找到数据传输性能问题吗?请让我知道是否需要 其他信息. Polybase运行ID:ad84913a-4b1e-4a75-8b62-19595c309dae BulkCopy运行ID:fe861b67-4195-4552-8c86-68d2185dc197和1db2f71b-c6f6-4531-962a-b51354455de3

I am facing performance issue when using BulkCopy From Blob Storage to SQL DW, I was doing troubleshoot since couple of days, but no result as of now. But when I use Polybase in Copy Data activity, it is very faster and completing in seconds/minutes instead of hours. I am using high configuration as below, still it is not helpful. Azure SQL DW – DWU – 1500 Copy Activity – Data Integration Unit – 16 Copy Activity – Degree of copy Parallelism – 20 When I use Polybase, I cant find a way to log the rejected rows in some location. Can you please help in finding the data transfer performance problem. Please let me know if you need any other information. Polybase Run Id: ad84913a-4b1e-4a75-8b62-19595c309dae BulkCopy Run Id: fe861b67-4195-4552-8c86-68d2185dc197 and 1db2f71b-c6f6-4531-962a-b51354455de3

推荐答案

我们遇到了类似的性能问题，但改用了Polybase，这要快得多.如果没有ploybase，我认为插入内容会通过DW的前门(通过主节点)，而polybase会跳过这些内容并知道如何直接复制数据 到DW中的右节点.

We had similar performance issues but switched to using Polybase which was a lot quicker. Without ploybase I think the inserts go through the front door in the DW (via the master node), while polybase some how skips this and knows how to directly copy data to the right node in the DW.

为什么不能使用polybase?

Why can't you use polybase?

这篇关于Azure数据工厂复制活动的性能问题！的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！