如何使用 terraform 添加 EMR 步骤 [英] How to add EMR steps using terraform

问题描述

在使用 terraform 创建 EMR 时，我需要通过 terraform 添加 EMR 步骤，因为我需要禁用 "keep_job_flow_alive_when_no_steps" .我该怎么做呢

I need to add EMR steps via terraform when creating my EMR using terraform since I need to disable "keep_job_flow_alive_when_no_steps" . How can I do that

推荐答案

要向 EMR 集群添加步骤，您可以在 EMR 资源中添加步骤部分.

To add steps to your EMR cluster you can add a steps section in your EMR resource.

如果您的my-bootstrap.sh"文件中有以下行以从 S3 下载步骤 bash 文件:

Provided you have the following lines in your "my-bootstrap.sh" file to download a steps bash file from S3:

aws s3 cp s3://my-bucket/steps.sh /tmp/steps.sh
chmod +x /tmp/steps.sh

您可以使用以下 aws_emr_cluster 配置启动引导操作和步骤:

You could launch both bootstrap actions and steps with the following aws_emr_cluster configuration:

resource "aws_emr_cluster" "my-emr-cluster" {
  name                              = "my-emr-cluster"
  release_label                     = "emr-5.9.0"

  ...

  bootstrap_action {
    path = "s3://my-bucket/my-bootstrap.sh"
    name = "my-emr-bootstrap"
  }

  step {
    action_on_failure = "CANCEL_AND_WAIT"
    name   = "Download S3 files to HDFS"
    hadoop_jar_step {
      jar  = "command-runner.jar"
      args = ["bash", "/tmp/steps.sh"]
    }
  }

  ...

}

此外，正如 Fionn 所说，如果您只想启用/禁用 keep_job_flow_alive_when_no_steps，则无需添加步骤.

Also, and as Fionn says, there is no need to add steps if you just want to enable/disable keep_job_flow_alive_when_no_steps.

这篇关于如何使用 terraform 添加 EMR 步骤的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！