Zookeeper/SASL校验和失败 [英] Zookeeper/SASL Checksum failed

问题描述

如何解决产生此错误的问题:

How do I fix the problem that generates this error:

WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@1040] - Client failed to SASL authenticate: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)]
javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)]
    at com.sun.security.sasl.gsskerb.GssKrb5Server.evaluateResponse(GssKrb5Server.java:199)
    at org.apache.zookeeper.server.ZooKeeperSaslServer.evaluateResponse(ZooKeeperSaslServer.java:50)

我已经在AWS EC2实例上设置了Zookeeper.我已经在此处概述了设置Kerberos和Zookeeper的步骤. Zookeeper似乎在工作:

I have set up Zookeeper on an AWS EC2 instance. I have outlined the steps I followed to set up Kerberos and Zookeeper here. Zookeeper seems to be working:

zookeeper@zookeeper-server-01:~/zk/zookeeper-3.4.11$ JVMFLAGS="-Djava.security.auth.login.config=/home/zookeeper/jaas/jaas.conf -Dsun.security.krb5.debug=true" bin/zkServer.sh start-foreground
...
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsRep cons in KrbAsReq.getReply zookeeper/zookeeper-server-01
2017-12-22 00:21:52,308 [myid:] - INFO  [main:Login@297] - Server successfully logged in.
2017-12-22 00:21:52,312 [myid:] - INFO  [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181
2017-12-22 00:21:52,313 [myid:] - INFO  [Thread-1:Login$1@130] - TGT refresh thread started.
2017-12-22 00:21:52,313 [myid:] - INFO  [Thread-1:Login@305] - TGT valid starting at:        Fri Dec 22 00:21:52 UTC 2017
2017-12-22 00:21:52,313 [myid:] - INFO  [Thread-1:Login@306] - TGT expires:                  Fri Dec 22 10:21:52 UTC 2017
2017-12-22 00:21:52,314 [myid:] - INFO  [Thread-1:Login$1@185] - TGT refresh sleeping until: Fri Dec 22 08:25:59 UTC 2017

但是，当我尝试将zkCli.sh(在另一个EC2实例上运行)连接到它时，服务器关闭连接并输出上面的校验和错误.

When I try, however, to connect a zkCli.sh (running on a different EC2 instance) to it, the server closes the connection and outputs the checksum error above.

Zookeeper客户端似乎能够连接到Zookeeper服务器:

The Zookeeper client appears to be able to connect to the Zookeeper server:

JVMFLAGS="-Djava.security.auth.login.config=/home/admin/Downloads/zookeeper-3.4.11/conf/zookeeper-test-client-jaas.conf -Dsun.security.krb5.debug=true" bin/zkCli.sh -server zookeeper-server-01.eigenroute.com:2181
Connecting to zookeeper-server-01.eigenroute.com:2181
2017-12-22 00:27:12,779 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=
3.4.11-37e277162d567b55a07d1755f0b31c32e93c01a0, built on 11/01/2017 18:06 GMT
...
2017-12-22 00:27:12,788 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/home/admin/Downloads/zookeeper-3.4.11
2017-12-22 00:27:12,789 [myid:] - INFO  [main:ZooKeeper@441] - Initiating client connection, connectString=zookeeper-server-01.eigenroute.com:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@1de0aca6
Welcome to ZooKeeper!
JLine support is enabled
...
>>> KrbAsReq creating message
[zk: zookeeper-server-01.eigenroute.com:2181(CONNECTING) 0] >>> KrbKdcReq send: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000, number of retries =3, #bytes=166
>>> KDCCommunication: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000,Attempt =1, #bytes=166
>>> KrbKdcReq send: #bytes read=310
>>>Pre-Authentication Data:
...

客户端收到有关需要预授权的错误，但随后似乎已成功登录到...(是否表示已成功通过身份验证?)... Zookeeper服务器?还是登录Kerberos?:

The client receives an error about needing preauthorization, but then appears to be successfully logged in (does this mean successfully authenticated?) to ...the Zookeeper server? Or logged into Kerberos?:

...
KRBError received: NEEDED_PREAUTH
KrbAsReqBuilder: PREAUTH FAILED/REQ, re-send AS-REQ
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23.
Looking for keys for: zktestclient/[e-mail]
Added key: 17version: 3
Added key: 18version: 3
Looking for keys for: zktestclient/[e-mail]
Added key: 17version: 3
Added key: 18version: 3
Using builtin default etypes for default_tkt_enctypes
default etypes for default_tkt_enctypes: 18 17 16 23.
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsReq creating message
>>> KrbKdcReq send: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000, number of retries =3, #bytes=253
>>> KDCCommunication: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000,Attempt =1, #bytes=253
>>> KrbKdcReq send: #bytes read=742
>>> KdcAccessibility: remove kerberos-server-01.eigenroute.com
Looking for keys for: zktestclient/[e-mail]
Added key: 17version: 3
Added key: 18version: 3
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbAsRep cons in KrbAsReq.getReply zktestclient/eigenroute.com
2017-12-22 00:27:13,286 [myid:] - INFO  [main-SendThread(35.169.37.216:2181):Login@297] - Client successfully logged in.
...

然后，客户端打开与Zookeeper服务器的套接字连接，并尝试对其进行SASL身份验证:

The client then opens a socket connection to the Zookeeper server, and attempts to SASL authenticate to it:

...
2017-12-22 00:27:13,312 [myid:] - INFO  [main-SendThread(35.169.37.216:2181):ClientCnxn$SendThread@103
5] - Opening socket connection to server 35.169.37.216/35.169.37.216:2181. Will attempt to SASL-authen
ticate using Login Context section 'Client'
2017-12-22 00:27:13,317 [myid:] - INFO  [main-SendThread(35.169.37.216:2181):ClientCnxn$SendThread@877
] - Socket connection established to 35.169.37.216/35.169.37.216:2181, initiating session
2017-12-22 00:27:13,359 [myid:] - INFO  [main-SendThread(35.169.37.216:2181):ClientCnxn$SendThread@1302] - Session establishment complete on server 35.169.37.216/35.169.37.216:2181, sessionid = 0x1000436873a0001, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
Found ticket for zktestclient/[e-mail] to go to krbtgt/EIGENROUTE.COM@EIGENROUTE.
COM expiring on Fri Dec 22 10:27:13 UTC 2017
Entered Krb5Context.initSecContext with state=STATE_NEW
Found ticket for zktestclient/[e-mail] to go to krbtgt/EIGENROUTE.COM@EIGENROUTE.
COM expiring on Fri Dec 22 10:27:13 UTC 2017
Service ticket not found in the subject
>>> Credentials acquireServiceCreds: same realm
Using builtin default etypes for default_tgs_enctypes
default etypes for default_tgs_enctypes: 18 17 16 23.
>>> CksumType: sun.security.krb5.internal.crypto.RsaMd5CksumType
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbKdcReq send: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000, number of retries =3, #bytes=712
>>> KDCCommunication: kdc=kerberos-server-01.eigenroute.com UDP:88, timeout=30000,Attempt =1, #bytes=712
>>> KrbKdcReq send: #bytes read=678
>>> KdcAccessibility: remove kerberos-server-01.eigenroute.com
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
>>> KrbApReq: APOptions are 00000000 00000000 00000000 00000000
>>> EType: sun.security.krb5.internal.crypto.Aes256CtsHmacSha1EType
Krb5Context setting mySeqNumber to: 50687702
Krb5Context setting peerSeqNumber to: 0
Created InitSecContextToken:
0000: 01 00 6E 82 02 6B 30 82   02 67 A0 03 02 01 05 A1  ..n..k0..g......
...
0260: 33 25 94 1F 60 93 E9 CF   7E EF 15 82 F8 6D ED 06  3%..`........m..
0270: 43                                                 C

2017-12-22 00:27:13,405 [myid:] - INFO  [main-SendThread(35.169.37.216:2181):ClientCnxn$SendThread@1161] - Unable to read additional data from server sessionid 0x1000436873a0001, likely server has closed socket, closing socket connection and attempting reconnect

WATCHER::

WatchedEvent state:Disconnected type:None path:null

因此，SASL身份验证并非完全失败，但是Zookeeper服务器关闭了连接(由于校验和失败).

So SASL authentication is not a complete failure, but the Zookeeper server closes the connection (on account of a checksum failure).

更新#1.为了回应T-Heron的评论，客户端计算机上的nslookup zookeeper-server-01.eigenroute.com结果为:

UPDATE #1. In response to T-Heron's comment, the result of nslookup zookeeper-server-01.eigenroute.com on the client machine is:

Server:     172.31.0.2
Address:    172.31.0.2#53

Non-authoritative answer:
Name:   zookeeper-server-01.eigenroute.com
Address: 35.169.37.216

zookeeper-server-01.eigenroute.com的DNS条目是:

zookeeper-server-01.eigenroute.com  30 minutes  A       
35.169.37.216

在客户端计算机上，/etc/hosts包含:

On the client machine, /etc/hosts contains:

127.0.1.1 ip-172-31-95-211.ec2.internal ip-172-31-95-211
127.0.0.1 localhost
34.239.197.36 kerberos-server-02

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

(kerberos-server-02的命名错误，它不是KDC，当我注释此行时得出的结果是相同的)，并且在ZooKeeper服务器上，zookeeper-server-01.eigenroute.com，/etc/hosts包含:

(kerberos-server-02 is misnamed, it is not a KDC, when I comment this line out the result is the same) and on the ZooKeeper server, zookeeper-server-01.eigenroute.com, /etc/hosts contains:

127.0.1.1 ip-172-31-88-14.ec2.internal ip-172-31-88-14
127.0.0.1 localhost
34.225.180.212 kerberos-server-01

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

(kerberos-server-01的条目不需要在那里-当我删除它时，结果是相同的).

(the entry for kerberos-server-01 doesn't need to be there - when I remove it the result is the same).

有人可以解释如何解决校验和失败吗?谢谢！

Can someone explain how to solve the checksum failure? Thanks!

推荐答案

我的KDC具有以下原则:

My KDC had the following principals:

zookeeper/[e-mail]
zookeeper/[e-mail]

在ZooKeeper服务器的JAAS配置中，该服务器的主机名是zookeeper-server-01.eigenroute.com，我使用了为zookeeper/[e-mail]创建的密钥表.

In the JAAS configuration for the ZooKeeper server, whose host name is zookeeper-server-01.eigenroute.com, I used a keytab that I created for zookeeper/[e-mail].

当我改为为zookeeper/[e-mail]创建一个密钥表并将其用于ZooKeeper服务器的JAAS配置中时，一切正常-客户端的SASL身份验证成功.

When I instead created a keytab for zookeeper/[e-mail] and used this keytab in the JAAS configuration for the ZooKeeper server, everything worked - SASL authentication from the client succeeded.

我宁愿在Kerberos主体的名称中使用完全限定的域名(zookeeper-server-01.eigenroute.com)，而不是IP地址.如果有人可以告诉我如何使它工作，我将接受它作为答案.在那之前，就足够了.

I would rather use the fully qualified domain name (zookeeper-server-01.eigenroute.com) in the name of the Kerberos principal, rather than the IP address. If anyone can tell me how to get that working, I'll accept that as the answer. Until then, this will suffice.

更新:我知道了. Zookeeper客户端从-server参数获取FQDN，查找该FQDN的IP地址，并从此(org.apache.zookeeper.client.StaticHostProvider)创建InetSocketAddress对象.然后，要获取主机名，它将调用.getHostName(org.apache.zookeeper.ClientCnxn.SendThread.startConnect).在我的本地计算机上，返回:

UPDATE: I figured it out. The Zookeeper client takes the FQDN from the -server argument, looks up the IP Address of this FQDN, and creates an InetSocketAddress object from this (org.apache.zookeeper.client.StaticHostProvider). Then to get the host name, it calls .getHostName (org.apache.zookeeper.ClientCnxn.SendThread.startConnect). On my local machine, this returns:

ec2-35-169-37-216.compute-1.amazonaws.com

在我的客户端AWS EC2实例上，返回:

and on my client AWS EC2 instance, this returns:

35.169.37.216

相反，我希望它返回FQDN.这就是为什么在我的AWS EC2客户端计算机上，ZooKeeper客户端尝试获取以下票证的原因:

when instead I expected it to return the FQDN. This is why on my AWS EC2 client machine, the ZooKeeper client tries to get a ticket for:

zookeeper/[e-mail]

在我的本地计算机上，ZooKeeper客户端尝试获取票证:

and on my local machine, the ZooKeeper client tries to get a ticket for:

zookeeper/[e-mail]

因此，我需要AWS确保在35.169.37.216上进行反向DNS查找会产生zookeeper-server-01.eigenroute.com.我到目前为止找到的解决方案是询问AWS设置反向DNS的映射.

So I need AWS to make sure that a reverse DNS lookup on 35.169.37.216 yields zookeeper-server-01.eigenroute.com. The solution I found so far is to ask AWS to set up the mapping for the reverse DNS.

理想情况下，ZooKeeper可以选择跳过此反向DNS查找，而仅使用FQDN作为主机名(也许可以，但我还没有找到).

Ideally, ZooKeeper would have an option to skip this reverse DNS lookup and just use the FQDN as the host name (maybe it does and I haven't found it).

这篇关于Zookeeper/SASL校验和失败的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！