我如何将Kerberos票证传递给Spring Yarn应用程序 [英] How can I pass a Kerberos ticket to Spring Yarn application

问题描述

我正在尝试运行简单的单一项目纱线应用程序，详细此处 。我将应用程序作为jar文件部署到我们的hadoop集群。试图运行时，我收到了一个异常，下面是堆栈跟踪：

  [2015-06-04 14：10：45.866 ]引导 -  13669 ERROR [主] --- SpringApplication：应用程序启动失败
 java.lang.IllegalStateException：未能执行在org.springframework.boot.SpringApplication.runCommandLineRunners CommandLineRunner 
（SpringApplication.java:680 ）
处org.springframework.boot.SpringApplication.run（SpringApplication.java:322）
 org.springframework.boot.SpringApplication.afterRefresh（SpringApplication.java:695）
。在组织。 springframework.boot.SpringApplication.run（SpringApplication.java:961）
处com.aetna.ise.yarn.publish org.springframework.boot.SpringApplication.run（SpringApplication.java:950）
。 Application.main（Application.java:21）
 at sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method）
 at sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:95 ）
在sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:56）
处org.springframework java.lang.reflect.Method.invoke（Method.java:620）
。 boot.loader.MainMethodRunner.run（MainMethodRunner.java:53）
在java.lang.Thread.run（Thread.java:857）
产生的原因：org.springframework.yarn.YarnSystemException：简单认证未启用。可用：[TOKEN，KERBEROS];嵌套异常是org.apache.hadoop.security.AccessControlException：SIMPLE身份验证未启用。可用：[TOKEN，KERBEROS] 
  

这是由于我们的集群使用Kerberos身份验证。有没有办法将Kerberos票据传递给Spring YARN代码中的应用程序？我没有看到有任何地方可以这样做。 解决方案

应用程序本身可以使用Kerberos。

这在 http://docs.spring.io/spring-hadoop/docs/2.1 .2.RELEASE / reference / html / springandhadoop-security.html＃literal-spring-hadoop-security-literal-configuration-properties

例如，如application.yml中所示（使用集群中的主体）：

  spring：
 hadoop：
 fsUri：HDFS：//本地主机：8020 
 resourceManagerHost：本地主机
安全：
 userPrincipal：jvalkealahti /新
 userKeytab：/usr/local/hadoops/jvalkealahti.keytab 
 authMethod：kerberos 
 namenodePrincipal：hdfs / neo @ LOCALDOMAIN 
 rmManagerPrincipal：yarn / neo @ LOCALDOMAIN 
  

I am trying to run the Simple Single Project Yarn Application detailed here. I deployed the application as a jar file to our hadoop cluster. When trying to run, I am getting an exception, stack trace below:

[2015-06-04 14:10:45.866] boot - 13669 ERROR [main] --- SpringApplication: Application startup failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
        at org.springframework.boot.SpringApplication.runCommandLineRunners(SpringApplication.java:680)
        at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:695)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:322)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:961)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:950)
        at com.aetna.ise.yarn.publish.Application.main(Application.java:21)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
        at java.lang.reflect.Method.invoke(Method.java:620)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:53)
        at java.lang.Thread.run(Thread.java:857)
Caused by: org.springframework.yarn.YarnSystemException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]; nested exception is org.apache.hadoop.security.AccessControlException: SIMPLE authentication is not enabled.  Available:[TOKEN, KERBEROS]

This is due to the fact that our cluster uses Kerberos authentication. Is there a way to pass the Kerberos ticket to the application in the Spring YARN code? I don't see any place to do that.

解决方案

We can't currently delegate any tickets when application is submitted, but application itself can use kerberos.

This is explained in section http://docs.spring.io/spring-hadoop/docs/2.1.2.RELEASE/reference/html/springandhadoop-security.html#literal-spring-hadoop-security-literal-configuration-properties

For example something like shown below in application.yml(use principals from your cluster):

spring:
  hadoop:
    fsUri: hdfs://localhost:8020
    resourceManagerHost: localhost
    security:
      userPrincipal: jvalkealahti/neo
      userKeytab: /usr/local/hadoops/jvalkealahti.keytab
      authMethod: kerberos
      namenodePrincipal: hdfs/neo@LOCALDOMAIN
      rmManagerPrincipal: yarn/neo@LOCALDOMAIN

这篇关于我如何将Kerberos票证传递给Spring Yarn应用程序的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！