运行 bash 脚本作为 EMR 中的一个步骤的正确语法是什么? [英] What is the correct syntax for running a bash script as a step in EMR?

问题描述

作为 EMR 完成引导后的一个步骤，我正在尝试运行 bash 脚本.以下是我的地形代码:

step { action_on_failure = "CONTINUE"name = "设置 Hadoop 配置"hadoop_jar_step {jar = "command-runner.jar"args = ["bash,-c,'cd/mnt; chmod +x ./userdata.sh; ./userdata.sh'"]}}

这不起作用，并且一直没有提到找不到文件.args 是否正确提供?script-runner.jar 呢?任何帮助都受到高度赞赏.:(

解决方案

使用 command-runner.jar 你可以执行很多程序，比如 bash 脚本，并且您不必像 script-runner.jar 那样知道它的完整路径.推荐使用command-runner.jar.

EMR是集群模式，不知道是哪个节点执行shell脚本，所以push到S3:

<块引用>

<代码>{"Name": "设置 Hadoop 配置","ActionOnFailure": "继续",HadoopJarStep":{"jar": "command-runner.jar",参数":["巴刹",-C"，" aws s3 cp s3://path_to_bucket_S3/userdata.sh .;chmod +x 用户数据.sh;./userdata.sh 参数...;rm 用户数据.sh"]}}

I am trying to run a bash script as a step after EMR completes bootstrapping. Following is my terraform code:

step { action_on_failure = "CONTINUE" 
name = "Setup Hadoop configuration" 
hadoop_jar_step { 
jar = "command-runner.jar" 
args = ["bash,-c,'cd /mnt; chmod +x ./userdata.sh; ./userdata.sh'"] 
}}

This isn't working and keeps on failing mentioning that No file found. Are args correctly provided ? What about script-runner.jar? Any help is highly appreciated. :(

解决方案

With command-runner.jar you can execute many programs like bash script, and you do not have to know its full path as was the case with script-runner.jar. It is recommended to use command-runner.jar.

EMR is in cluster mode, you do not know which nodes execute the shell script, so push it to S3 :

{
  "Name": "Setup Hadoop configuration",
  "ActionOnFailure": "CONTINUE",
 "HadoopJarStep": {
    "Jar": "command-runner.jar",
    "Args": [
         "bash",
         "-c",
         " aws s3 cp s3://path_to_bucket_S3/userdata.sh .;
          chmod +x userdata.sh;
          ./userdata.sh args...;
          rm userdata.sh "
     ]
  }
}

这篇关于运行 bash 脚本作为 EMR 中的一个步骤的正确语法是什么?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！