如何在没有kerberos的情况下启动spark 3.0.0 kubernetes工作负载? [英] How to launch spark 3.0.0 kubernetes workload without kerberos?

问题描述

似乎在spark 3.0.0上，当我使用kubernetes进行spark提交时，它需要使用kerberos，我使用在2.4.5中运行良好的相同spark提交. 我收到此错误:

It seems that on spark 3.0.0 when I do a spark submit with kubernetes it require kerberos, I use the same spark submit that was working great in 2.4.5 I get this error:

Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
20/07/04 08:17:51 INFO SparkKubernetesClientFactory: Auto-configuring K8S client using current context from users K8S config file
20/07/04 08:17:51 INFO KerberosConfDriverFeatureStep: You have not specified a krb5.conf file locally or via a ConfigMap. Make sure that you have the krb5.conf locally on the driver image.
Exception in thread "main" org.apache.hadoop.security.KerberosAuthException: failure to login: javax.security.auth.login.LoginException: java.lang.NullPointerException: invalid null input: name

更具体地说，我不想使用kerberos

To be more specific, I don't want to use kerberos

推荐答案

您可以在操作系统级别处理此ID(185)，也可以在dockerfile中对其进行注释，但您将是root.... 埃里克

either you handle this id ( 185) at os level, or you can comment it in the dockerfile but you will be root.... Eric

这篇关于如何在没有kerberos的情况下启动spark 3.0.0 kubernetes工作负载?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！