如何使用spark-submit为Spark作业选择队列? [英] How to choose the queue for Spark job using spark-submit?

问题描述

是否可以提供参数或设置来选择希望运行spark_submit作业的队列?

Is there a way to provide parameters or settings to choose the queue in which I'd like my spark_submit job to run?

推荐答案

通过使用--queue 因此，火花提交作业的示例为:-

By using --queue So an example of a spark-submit job would be:-

spark-submit --master yarn --conf spark.executor.memory = 48G --conf spark.driver.memory = 6G --packages [以，分隔的软件包] --queue [queue_name] --class [class_name ] [jar_file] [参数]

spark-submit --master yarn --conf spark.executor.memory=48G --conf spark.driver.memory=6G --packages [packages separated by ,] --queue [queue_name] --class [class_name] [jar_file] [arguments]

这篇关于如何使用spark-submit为Spark作业选择队列?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！