如何引用在DAG之外创建的sys.stdout，以便与withParam一起在DAG内使用? [英] How to reference sys.stdout created outside of a DAG to be used inside of a DAG with withParam?

问题描述

我正在使用Argo工作流程.

I am working with an Argo workflow.

我的 entrypoint 中有一个DAG步骤，该步骤遵循几个常规步骤.这些步骤之一是执行 sys.stdout .进入DAG步骤之后，我希望某些任务引用 sys.stdout 中的结果.

There is a DAG step in my entrypoint which follows several normal steps. One of these steps does a sys.stdout. Once inside of the DAG step, I want some of the tasks to reference the results from the sys.stdout.

我知道，当工作流程从一个步骤转到另一个步骤(不使用DAG)时，是否要引用 sys.stdout ，我们可以执行 {{steps.step-name.outputs.result}} .但是，在DAG任务中，这是行不通的.

I know if we wanted to reference the sys.stdout when the workflow just goes from one step to the next (without the DAG), we can do {{steps.step-name.outputs.result}}. The same does not work inside of a DAG task though.

如何在DAG任务中引用sys.stdout，以便与 withParam 一起使用?

How can I reference the sys.stdout inside of a DAG task so I can use it with withParam?

工作流程如下:

  templates:
  - name: the-entrypoint
    steps:
    - - name: step01
        template: first-step
    - - name: step02
        template: second-step
    - - name: step03
        template: third-step
    - - name: step04-the-dag-step
        template: fourth-step

通常，如果 third-step 执行 sys.stdout ，我们可以通过 {{steps.step03.outputs.result}}} 进行第四步.但是，在这种情况下， fourth-step 是DAG，并且如果DAG任务之一想使用 sys.stdout ，则调用 {{steps.step03.outputs.result}} 作为DAG任务中的参数/参数会引发错误.

In general, if third-step does a sys.stdout, we can reference it by {{steps.step03.outputs.result}} in fourth-step. However, in this case fourth-step is a DAG, and if one of the DAG tasks wants to use the sys.stdout, calling {{steps.step03.outputs.result}} as an argument/parameter inside of DAG tasks throws up an error.

然后的问题是，如何在DAG第四步任务中正确引用第三步生成的 sys.stdout ?/p>

The question is then how can one correctly reference the sys.stdout generated by third-step inside fourth-step DAG tasks?

推荐答案

关于模板输出的一些背景知识

Argo Workflows支持许多不同的每种类型的模板在模板中都支持不同类型的引用 .

Each type of template supports different types of reference within the template.

在 steps 模板中，您可以使用 steps.step-name.outputs.parameters.param-name 来访问其他步骤的输出参数code>(用于命名参数)或 steps.step-name.outputs.result (用于 script 或 container 模板的标准输出).

Within a steps template, you may access the output parameters of other steps with steps.step-name.outputs.parameters.param-name (for named parameters) or steps.step-name.outputs.result (for the stdout of a script or container template).

示例(查看完整的工作流程):

  - name: output-parameter
    steps:
    - - name: generate-parameter
        template: whalesay
    - - name: consume-parameter
        template: print-message
        arguments:
          parameters:
          - name: message
            value: "{{steps.generate-parameter.outputs.parameters.hello-param}}"

在 dag 模板中，您可以使用类似的语法访问各种任务的输出，而只需使用 tasks.而不是步骤..

示例(查看完整的工作流程):

    - name: main
      dag:
        tasks:
          - name: flip-coin
            template: flip-coin
          - name: heads
            depends: flip-coin
            template: heads
            when: "{{tasks.flip-coin.outputs.result}} == heads"
          - name: tails
            depends: flip-coin
            template: tails
            when: "{{tasks.flip-coin.outputs.result}} == tails"

在容器或 script 模板中，您只能访问该模板的输入 *.您可能无法直接从容器或脚本模板访问步骤或任务模板中的步骤或任务的输出.

Within a container or script template, you may access only the inputs of that template*. You may not directly access the outputs of steps or tasks from steps or tasks templates from a container or script template.

如上所述，DAG模板无法直接引用 steps 模板中的步骤输出.但是 steps 模板中的某个步骤可以将步骤输出传递到DAG模板.

As mentioned above, a DAG template cannot directly reference step outputs from a steps template. But a step within a steps template can pass a step output to a DAG template.

在您的示例中，它看起来像这样:

In your example, it would look something like this:

  templates:
  - name: the-entrypoint
    steps:
    - - name: step01
        template: first-step
    - - name: step02
        template: second-step
    - - name: step03
        template: third-step
    - - name: step04-the-dag-step
        template: fourth-step
        arguments:
          parameters:
          - name: some-param
            value: "{{steps.step03.outputs.result}}"
  - name: fourth-step
    inputs:
      parameters:
      - name: some-param
    dag:
      tasks:
        # use the input parameter in the fourth-step template with "{{inputs.parameters.some-param}}"

tl; dr

步骤.和任务.变量是在单个步骤或任务模板中引用的，但是可以明确地指定它们在模板之间传递.如果需要在DAG中使用步骤的输出，请将该输出直接作为调用DAG的参数传递.

tl;dr

steps. and tasks. variables are meant to be referenced within a single steps- or tasks-template, but they can be explicitly passed between templates. If you need to use the output of a step in a DAG, directly pass that output as an argument where the DAG is invoked.

在您的情况下，将在四个步骤的最后一个步骤中调用DAG模板，因此您将在其中传递参数.

In your case, the DAG template is invoked as the last of four steps, so that is where you will pass the argument.

_{*好的，您还可以访问}脚本或 container 模板中的其他各种变量，但是您无权访问在另一个模板中作为内部变量限定范围的变量.

_{* Okay, you also have access to various other variables from within a script or container template, but you don't have access to variables that are scoped as internal variables within another template.}

这篇关于如何引用在DAG之外创建的sys.stdout，以便与withParam一起在DAG内使用?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！