通过编程获取ADF管道消耗报告 [英] Programatically Get ADF pipeline consumption report
本文介绍了通过编程获取ADF管道消耗报告的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我感兴趣的是查询Data Factory监视器提供的管道消耗报告。在Log Analytics或PowerShell cmdlet上是否有将返回此信息的表?我检查了ADFv2 PowerShell模块,但没有找到任何模块。我的目标是汇总本报告中提供的信息,以确定哪些是成本最高的管道。
谢谢
推荐答案
在做更多研究时,有人将我指向GitHub页面,产品团队在那里发布了一个PowerShell脚本,以查找我正在寻找的部分内容{1}。因此,我对脚本进行了一些修改,以获得所需的输出。有了下面的输出,我可以从MS计算器中提取值,以获得每次管道运行的估计成本。{2}
$startTime = "21/6/2021 7:00:00"
$endTime = "21/6/2021 10:00:00"
$adf = '<data factory name>'
$rg = '<resrouce group name>'
$outputObj = @()
$pipelineRuns = Get-AzDataFactoryV2PipelineRun -ResourceGroupName $rg -DataFactoryName $adf -LastUpdatedAfter $startTime -LastUpdatedBefore $endTime
# loop through all pipelines and child activities to return billable information
foreach ($pipelineRun in $pipelineRuns) {
$activtiyRuns = Get-AzDataFactoryV2ActivityRun -ResourceGroupName $rg -DataFactoryName $adf -pipelineRunId $pipelineRun.RunId -RunStartedAfter $startTime -RunStartedBefore $endTime
foreach ($activtiyRun in $activtiyRuns) {
if ($null -ne $activtiyRun.Output -and $null -ne $activtiyRun.Output.SelectToken("billingReference.billableDuration")) {
$obj = @()
$obj = $activtiyRun.Output.SelectToken("billingReference.billableDuration").ToString() | ConvertFrom-Json
$obj | Add-Member -MemberType NoteProperty -Name activityType -value $activtiyRun.Output.SelectToken("billingReference.activityType").ToString()
$obj | Add-Member -MemberType NoteProperty -Name pipelineName -value $pipelineRun.PipelineName
$obj | Add-Member -MemberType NoteProperty -Name activtiyRuns -value $activtiyRuns.Count
$outputObj += $obj
}
else {}
}
}
# output aggregated result set as table
$groupedObj = $outputObj | Group-Object -Property pipelineName, activityType, meterType
$groupedObj | ForEach-Object {
$value = $_.name -split ', '
New-Object psobject -Property @{
activityType = $value[1];
meterType = $value[2];
pipelineName = $value[0];
executionHours = [math]::Round(($_.Group | Measure-object -Property duration -sum).Sum, 4)
orchestrationActivityRuns = $groupedObj.group.activtiyRuns[0]
}
} | Sort-Object -Property meterType | Format-Table
输出示例:
来自数据工厂监视器的消耗报告
引用:
- https://github.com/Azure/Azure-DataFactory/tree/main/SamplesV2/PastRunDetails#simple-script-that-prints--activity-level-run-details-in-45-day-range{1}
- https://azure.microsoft.com/en-us/pricing/calculator/?service=data-factory%2F{2}
这篇关于通过编程获取ADF管道消耗报告的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文