根据大小从文件夹中的Azure Data Factory抓取文件 [英] Azure Data Factory grab file from folder based on size

问题描述

我运行了一个复制活动，该活动使用http链接服务从网上拉出一个zip文件，然后将该zip提取到Azure blob存储容器中包含多个文件的文件夹中.我现在想做的是从新创建的文件夹中动态提取最大的文件，并通过数据流转换运行它，同时还通过ADF删除该文件夹.我正在尝试获取元数据"活动，该活动输出文件夹的子项.然后，输出通过'@activity('Get Metadata1').output.childItems'连接到ForEach活动.通过内部GetMetadata活动在ForEach设置的项目"中传递以获取文件大小.但这在检索文件大小时出错，这给了我.

{"errorCode":"3500"，消息":字段'大小'失败，错误:'Type = Microsoft.WindowsAzure.Storage.StorageException，Message =远程服务器返回错误:(404)未找到.，Source = Microsoft.WindowsAzure.Storage，" Type = System.Net.WebException，Message =远程服务器返回错误:(404)找不到.，Source = System，."failureType":"UserError"，"target":"Get Metadata2"，详细信息":[]}

是否无法获取文件夹的子项的文件大小?我正在关注此文档.

通过比较获得最大的文件大小，然后再次循环以获取最大的blob.

I ran a copy activity that used a http linked service to pull a zip file from an online and then extract the zip to a folder with multiple files within an Azure blob storage container. What I want to do now is dynamically pull the largest file from that newly created folder and run it through a data flow transformation while also deleting the folder through ADF. I am trying with a Get metadata activity that outputs the child items of the folder. The output is then connected to a ForEach activity with '@activity('Get Metadata1').output.childItems.' being passed in the Items of the ForEach setting with an inner GetMetadata activity to get the file sizes. But it errors on retrieving the file size giving me this..

{ "errorCode": "3500", "message": "Field 'size' failed with error: 'Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (404) Not Found.,Source=Microsoft.WindowsAzure.Storage,''Type=System.Net.WebException,Message=The remote server returned an error: (404) Not Found.,Source=System,'.", "failureType": "UserError", "target": "Get Metadata2", "details": [] }

Is it not possible to get the file sizes of a folder's child items?. I was following this documentation.

https://social.msdn.microsoft.com/Forums/azure/en-US/a83712ef-9a1a-4741-80b5-0e2ee8288ef5/get-child-items-size?forum=AzureDataFactory&prof=required

解决方案

You can also use logic app to get the max size file:

First, you can use List blobs to get blobs in your folder.

Get the largest file size by comparing, and then loop again to get the largest blob.

这篇关于根据大小从文件夹中的Azure Data Factory抓取文件的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！