Spark 作为另一个用户提交到纱线 [英] Spark submit to yarn as a another user
问题描述
是否可以将一个 spark 作业提交到一个纱线集群并选择，无论是使用命令行还是在 jar 中，哪个用户将拥有"该作业?
Is it possible to submit a spark job to a yarn cluster and choose, either with the command line or inside the jar, which user will "own" the job?
spark-submit 将从包含用户的脚本启动.
The spark-submit will be launch from a script containing the user.
PS:如果集群具有 kerberos 配置(并且脚本是密钥表)，是否还有可能?
PS: is it still possible if the cluster has a kerberos configuration (and the script a keytab) ?
推荐答案
对于非 kerberized 集群:export HADOOP_USER_NAME=zorro 在提交 Spark 作业之前将执行把戏.
如果您想在其余的 shell 脚本(或交互式 shell 会话)中恢复到默认凭据，请确保在之后取消设置 HADOOP_USER_NAME.
For a non-kerberized cluster: export HADOOP_USER_NAME=zorro before submitting the Spark job will do the trick.
Make sure to unset HADOOP_USER_NAME afterwards, if you want to revert to your default credentials in the rest of the shell script (or in your interactive shell session).
对于kerberized 集群，在不破坏其他作业/会话(可能取决于您的默认票证)的情况下模拟另一个帐户的干净方式就是这样线...
For a kerberized cluster, the clean way to impersonate another account without trashing your other jobs/sessions (that probably depend on your default ticket) would be something in this line...
export KRB5CCNAME=FILE:/tmp/krb5cc_$(id -u)_temp_$$
kinit -kt ~/.protectedDir/zorro.keytab zorro@MY.REALM
spark-submit ...........
kdestroy
这篇关于Spark 作为另一个用户提交到纱线的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
