Azure数据工厂：将表导出为具有大小上限的平面文件 [英] Azure Data Factory: Exporting a table into flat files with a size cap

问题描述

我正在尝试将大型表从Azure SQL PAAS DB导出到Azure blob存储。我需要将表导出为多个平面文件，文件大小最大。每个10 GB。有没有办法可以用ADF实现它？

I'm trying to export a large table from an Azure SQL PAAS DB to Azure blob storage. I need to export the table into multiple flat files with a max file size  of 10 GB each. Is there a way that I can achieve it using ADF?

推荐答案

你好Surya Palo，谢谢你的询问。 虽然您想要的不是Data Factory的内置功能，但有多种方法可以实现相同的结果。

Hello Surya Palo, and thank you for your inquiry.  While what you want is not an inbuilt feature of Data Factory, there are multiple ways to achieve the same result.

一个解决方案可能如下所示：

One solution could look like this:

1估计10 GB文件中可容纳的行数。 估计文件数量。  （压缩可选）

1 Estimate how many rows would fit into the 10 GB file.  Estimate how many files.  (compression optional)

2创建一个只提取X和Y之间行的查询

2 Create a query which will extracts only rows between X and Y

3将复制活动放在循环活动中。  ;循环将告诉复制活动X和Y的值以及文件名。



您可以通过基于公共属性分割成文件来为您的解决方案增加价值/ 。值 这将使未来的搜索更快。



如果您有更多问题/需要更多帮助或想法，请告诉我。

3 Place the copy activity inside a loop activity.  The loop will tell the copy activity the values of X and Y, and the filename.

You could add value to your solution by splitting into files based upon common attributes/ values.  This would make future searches faster.

Let me know if you have more questions / need more help or ideas.

这篇关于Azure数据工厂：将表导出为具有大小上限的平面文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！