使用Kerberos的Hadoop Web身份验证 [英] Hadoop Web Authentication using Kerberos

问题描述

我使用kerberos配置hadoop，一切正常，我可以浏览hdfs，提交作业等，但是没有通过http认证。我使用hadoop-0.20。 2在cdh3u2中，它支持HTTP SPNEGO。

core-site.xml中的HTTP认证相关配置如下：

 <！ -  HTTP网页控制台身份验证 - > 
<属性> 
< name> hadoop.http.filter.initializers< / name> 
< value> org.apache.hadoop.security.AuthenticationFilterInitializer< / value> 
< / property> 
 
<属性> 
<名称> hadoop.http.authentication.type< /名称> 
< value> kerberos< /值> 
< / property> 
 
<属性> 
<名称> hadoop.http.authentication.token.validity< / name> 
<值> 36000< /值> 
< / property> 
 
<属性> 
<名称> hadoop.http.authentication.signature.secret.file< / name> 
<值> / home / hadoop / hadoop / conf / http-secret-file< /值> 
< / property> 
 
<属性> 
<名称> hadoop.http.authentication.cookie.domain< / name> 
<值>< /值> 
< / property> 
 
<属性> 
<名称> hadoop.http.authentication.simple.anonymous.allowed< / name> 
<值> false< /值> 
< / property> 
 
<属性> 
< name> hadoop.http.authentication.kerberos.principal< / name> 
< value> HTTP/[e-mail]< /值> 
< / property> 
 
<属性> 
< name> hadoop.http.authentication.kerberos.keytab< / name> 
<值> /home/hadoop/hadoop/conf/http.keytab< /值> 
< / property> 
< / configuration> 
  

在启动过程中，http验证成功。

  2011-11-15 15:43:59,106 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler：已初始化，主体[HTTP / hz169-91.i.site.com @ b 
 
 
 
 
 
 
 
 
 $ /查看代码之后，我发现AuthenticationFilter在doFilter期间获得了空的标记，因此，认证开始（代码如下），但httpservletrequest中的授权为空，因此，每当我重新加载我的页面时，就会出现一个日志。 p> 
 
 
  2011-11-15 15：47：52,190 WARN org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler：SPNEGO starting 
  
 
 
 
 
 
 
  // org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler 
 public AuthenticationToken authenticate（HttpServletRequest request，final HttpServletResponse respon se）
抛出IOException，AuthenticationException {
 AuthenticationToken token = null; 
 String authorization = request.getHeader（KerberosAuthenticator.AUTHORIZATION）; 
 if（authorization == null ||！authorization.startsWith（KerberosAuthenticator.NEGOTIATE））{
 response.setHeader（KerberosAuthenticator.WWW_AUTHENTICATE，KerberosAuthenticator.NEGOTIATE）; 
 response.setStatus（HttpServletResponse.SC_UNAUTHORIZED）; 
 if（authorization == null）{
 LOG.warn（SPNEGO starting）; 
} else {
 LOG.warn（'+ KerberosAuthenticator.AUTHORIZATION +'不以''开头+ 
 KerberosAuthenticator.NEGOTIATE +'：{}，authorization）; 
} 
  
是否有配置错误，或者只是我的浏览器不支持SPNEGO。我在Ubuntu 11.04中使用了Chrome v16。
 
 
 
是否有人有线索来帮助我弄清楚它？ 
 
 
谢谢
解决方案
第一：Thankyou发布了一个关于如何为SPNNEGO配置Hadoop Web控制台的完整工作示例 - 一个很好的例子。
 
 
在修改配置文件的路径（我创建了 hadoop.http.authentication.signature.secret.file 通过从/ dev / random获得一些随机字节，我假设这是正确的，尽管我找不到任何支持该理论的文档）。
 
 
 Google Chrome支持SPNNEGO从版本6.0.472开始并转发。但是，似乎在Linux和OSX上，您必须将它传递给可以启用它的服务器列表如此处所述。因此，尝试在启动Chrome时向cmdline添加*  -  auth-server-whitelist =* example.com，* foobar.com， baz。
 
 
另一种调试方法是使用更简单的浏览器。如果您的curl具有GSS协商支持，我会推荐curl。通过运行curl来检查--version 
  $ curl --version 
 curl 7.19.7（i486-pc-linux -gnu）libcurl / 7.19.7 OpenSSL / 0.9.8k zlib / 1.2.3.3 libidn / 1.15 
协议：tftp ftp telnet dict ldap ldaps http文件https ftps 
特性：GSS协商IDN IPv6大文件NTLM SSL libz 
  
如果列表中包含 GSS-Negotiate 使用curl尝试访问例如namenode Web控制台：
  $ curl -v -u foo --negotiate http：/ /your.namenode.tld:50070 
  
询问主机密码时按Enter键。
 
 
 
这应该会让您更好地了解客户端和服务器之间的情况。
 
I configured hadoop using kerberos, everything works fine, I can browse hdfs, submit jobs, etc. But failed http web authentication.

I use hadoop-0.20.2 in cdh3u2, which supports HTTP SPNEGO.

HTTP authentication related configurations in core-site.xml are as follows:
  <!-- HTTP web-consoles Authentication -->
  <property>
    <name>hadoop.http.filter.initializers</name>
    <value>org.apache.hadoop.security.AuthenticationFilterInitializer</value>
  </property>

  <property>
    <name>hadoop.http.authentication.type</name>
    <value>kerberos</value>
  </property>

  <property>
    <name>hadoop.http.authentication.token.validity</name>
    <value>36000</value>
  </property>

  <property>
    <name>hadoop.http.authentication.signature.secret.file</name>
    <value>/home/hadoop/hadoop/conf/http-secret-file</value>
  </property>

  <property>
    <name>hadoop.http.authentication.cookie.domain</name>
    <value></value>
  </property>

  <property>
    <name>hadoop.http.authentication.simple.anonymous.allowed</name>
    <value>false</value>
  </property>

  <property>
    <name>hadoop.http.authentication.kerberos.principal</name>
    <value>HTTP/[e-mail]</value>
  </property>

  <property>
    <name>hadoop.http.authentication.kerberos.keytab</name>
    <value>/home/hadoop/hadoop/conf/http.keytab</value>
  </property>
</configuration>
During startup, http authentication succeeded.
2011-11-15 15:43:59,106 INFO org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: Initialized, principal [HTTP/[e-mail]] from keytab [/home/hadoop/hadoop/conf/http.keytab]
After look into the code, I found out that AuthenticationFilter gets null token during doFilter, so, authentication begins (code below), but authorization in httpservletrequest is null, so, every time I reload my page, one log appears.
2011-11-15 15:47:52,190 WARN org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler: SPNEGO starting




// org.apache.hadoop.security.authentication.server.KerberosAuthenticationHandler
public AuthenticationToken authenticate(HttpServletRequest request, final HttpServletResponse response)
    throws IOException, AuthenticationException {
    AuthenticationToken token = null;
    String authorization = request.getHeader(KerberosAuthenticator.AUTHORIZATION);
    if (authorization == null || !authorization.startsWith(KerberosAuthenticator.NEGOTIATE)) {
      response.setHeader(KerberosAuthenticator.WWW_AUTHENTICATE, KerberosAuthenticator.NEGOTIATE);
      response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
      if (authorization == null) {
        LOG.warn("SPNEGO starting");
      } else {
        LOG.warn("'" + KerberosAuthenticator.AUTHORIZATION + "' does not start with '" +
            KerberosAuthenticator.NEGOTIATE + "' :  {}", authorization);
      }
Is there any configuration errors, or just my browser doesn't support SPNEGO. I use Chrome v16 in Ubuntu 11.04.

Does anybody have clues to help me figure it out?

Thanks.
解决方案
First: Thankyou for posting a complete and working example on how to configure Hadoop web consoles for SPNNEGO - I had trouble finding a good example.

Your example works for me after modifying paths to config files (I created hadoop.http.authentication.signature.secret.file by getting some random bytes from /dev/random, which I'm assuming is the right thing to do, although I can't find any documentation supporting that theory).

Google Chrome does support SPNNEGO from version 6.0.472 and forward. However, it seems that on Linux and OSX you have to pass it a list of servers for which it's OK to enable it as documented here. So, try adding *--auth-server-whitelist="*example.com,*foobar.com,baz" to the cmdline when starting Chrome.

Another way of debugging this would be to use a simpler browser. I would recommend curl, if your curl has GSS-Negotiate support. Check by running curl --version
$ curl --version
curl 7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps 
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz 
If GSS-Negotiate is in the Features list, you can use curl to try to access for example the namenode Web Console:
$ curl -v -u foo --negotiate http://your.namenode.tld:50070
Just press enter when asked for host password.

This should give you a better idea on what's going on between the client and the server.

                        
这篇关于使用Kerberos的Hadoop Web身份验证的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！