Azure datafactory v2分别执行管道 [英] Azure datafactory v2 Execute Pipeline with For Each
问题描述
我正在尝试使用执行管道"来调用具有ForEach活动的管道.我收到错误消息.
I am trying to use "Execute Pipeline" to invoke a Pipe which has a ForEach activity. I get an error.
- 用于执行管道的Json:
[
{
"name": "pipeline3",
"properties": {
"activities": [
{
"name": "Test_invoke1",
"type": "ExecutePipeline",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"pipeline": {
"referenceName": "MAIN_SA_copy1",
"type": "PipelineReference"
},
"waitOnCompletion": true
}
}
],
"annotations": []
}
}
]
- Jason为每个活动调用管道:
[
{
"name": "MAIN_SA_copy1",
"properties": {
"activities": [
{
"name": "Collect_SA_Data",
"type": "ForEach",
"dependsOn": [],
"userProperties": [],
"typeProperties": {
"items": {
"value": "@pipeline().parameters.TableNames",
"type": "Expression"
},
"batchCount": 15,
"activities": [
{
"name": "Sink_SAdata_toDL",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [
{
"name": "Destination",
"value": "@{pipeline().parameters.DLFilePath}/@{item()}"
}
],
"typeProperties": {
"source": {
"type": "SqlServerSource",
"sqlReaderQuery": {
"value": "@concat('SELECT * FROM ',item())",
"type": "Expression"
}
},
"sink": {
"type": "AzureBlobFSSink"
},
"enableStaging": false,
"parallelCopies": 1,
"dataIntegrationUnits": 4
},
"inputs": [
{
"referenceName": "SrcDS_StructuringAnalytics",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "ADLS",
"type": "DatasetReference",
"parameters": {
"FilePath": "@pipeline().parameters.DLFilePath",
"FileName": {
"value": "@concat(item(),'.orc')",
"type": "Expression"
}
}
}
]
}
]
}
}
],
"parameters": {
"DLFilePath": {
"type": "string",
"defaultValue": "extracts/StructuringAnalytics"
},
"TableNames": {
"type": "array",
"defaultValue": [
"fom.FOMLineItem_manual"
]
}
},
"variables": {
"QryTableColumn": {
"type": "String"
},
"QryTable": {
"type": "String"
}
},
"folder": {
"name": "StructuringAnalytics"
},
"annotations": []
},
"type": "Microsoft.DataFactory/factories/pipelines"
}
]
我收到一个错误:
[
{
"errorCode": "BadRequest",
"message": "Operation on target Collect_SA_Data failed: The execution of template action 'Collect_SA_Data' failed: the result of the evaluation of 'foreach' expression '@pipeline().parameters.TableNames' is of type 'String'. The result must be a valid array.",
"failureType": "UserError",
"target": "Test_invoke1",
"details": ""
}
]
输入:
"pipeline": {
"referenceName": "MAIN_SA_copy1",
"type": "PipelineReference"
},
"waitOnCompletion": true,
"parameters": {
"DLFilePath": "extracts/StructuringAnalytics",
"TableNames": "[\"fom.FOMLineItem_manual\"]"
}
推荐答案
我猜您正在使用UI来设置管道及其参数，并且我希望您希望将被调用管道的数组参数放置在其他类似位置:(这完全是我的猜测，因为我做的完全一样，结果也一样)
I guess you were using the UI to set the pipeline and its parameters and I guess you expected to put the array parameter of the called pipeline as everywhere else like this: (It is all my guess, because I just did exactly the same, with the same result)
诀窍是在代码中定义数组(["table1"，"table2"]):
The trick is to define the array in the Code (["table1", "table2"]):
UI中的输入将如下所示:
The input in the UI will look like this:
现在可以使用了！
看来，Datafactory会将整个数组视为某个数组的一个元素.因此，有时可以使用array()函数的解决方案.
看起来像个错误，定义了数组参数输入.
Now it works!
It seems, that the Datafactory is otherwise treating the whole array as one element of some array. Hence, the solution with the array() function sometimes works.
It looks like a bug, defining array parameter input..
(必须编辑答案，我首先认为在UI输入中省略冒号就足够了)
(Had to edit the answer, I first thought omiting the colons in the UI input would be enough)
这篇关于Azure datafactory v2分别执行管道的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
