步骤函数中的AWS粘合作业依赖关系 [英] aws glue job dependency in step function
本文介绍了步骤函数中的AWS粘合作业依赖关系的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!
问题描述
我已创建2个胶水作业(Gluejob1、Gluejob2)。
我想创建一个依赖项,因为只有在glejob1完成之后,glejob2才应该运行。
为了协调这一过程,我创建了一个具有以下定义的步骤函数:
{
"gluejob1": {
"Type": "Task",
"Resource": "gluejob1.Arn",
"Comment": "Glue job1.",
"Next": "gluejob2"
},
"gluejob2": {
"Type": "Task",
"Resource": "gluejob2.Arn",
"Comment": "TGlue job2.",
"Next": "Gluejob2 Finished Loading"
},
"Gluejob2 Finished Loading": {
"Type": "Pass",
"Result": "",
"End": true
}
}
当我执行此步骤函数时,STATE函数在触发Gluejob1并继续触发Gluejob2时称其为成功。
我想知道是否可以仅在Gluejob1完成后运行Gluejob2。
推荐答案
您可以invoke Glue job from StepFunction synchronously使其等待作业完成:
{
"StartAt": "gluejob1",
"States": {
"gluejob1": {
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Parameters": {
"JobName.$": "ETLJobName1"
},
"Next": "gluejob2"
},
"gluejob2": {
"Type": "Task",
"Resource": "arn:aws:states:::glue:startJobRun.sync",
"Parameters": {
"JobName.$": "ETLJobName2"
},
"Next": "Gluejob2 Finished Loading"
},
"Gluejob2 Finished Loading": {
"Type": "Pass",
"Result": "",
"End": true
}
}
这篇关于步骤函数中的AWS粘合作业依赖关系的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!
查看全文