Boto3 EMR-蜂巢步骤 [英] Boto3 EMR - Hive step

问题描述

是否可以使用Boto 3执行蜂巢步骤?我一直在使用AWS CLI，但是从文档(

Is it possible to carry out hive steps using boto 3? I have been doing so using AWS CLI, but from the docs (http://boto3.readthedocs.org/en/latest/reference/services/emr.html#EMR.Client.add_job_flow_steps), it seems like only jars are accepted. If Hive steps are possible, where are the resources?

谢谢

推荐答案

我能够使用Boto3使它起作用:

I was able to get this to work using Boto3:

# First create your hive command line arguments
hive_args = "hive -v -f s3://user/hadoop/hive.hql"

# Split the hive args to a list
hive_args_list = hive_args.split()

# Initialize your Hive Step 
hiveEmrStep=[
        {
            'Name': 'Hive_EMR_Step',
            'ActionOnFailure': 'CONTINUE',
            'HadoopJarStep': {
                'Jar': 'command-runner.jar',
                'Args': hive_args_list
            }
        },
    ]

# Create Boto3 session and client
session = boto3.Session(region_name=AWS_REGION,profile_name=AWS_PROFILE)
client = session.client('emr')

# Submit and execute EMR Step
client.add_job_flow_steps(JobFlowId=cluster_id,Steps=hiveEmrStep)

#Where cluster_id is the ID of your cluster from AWS EMR (ex: j-2GS7xxxxxx)

这篇关于Boto3 EMR-蜂巢步骤的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！