使用Kerberos从Windows连接到Unix上的Kafka [英] Connect to Kafka on Unix from Windows with Kerberos

问题描述

我对Kafka还是很陌生，所以请多多包涵.这是我的设置. 我在统一框上托管了kafka.聚集.在域中说B. 客户端在Windows上.并尝试使用域A连接到托管在B上的kafka. 我有密钥表.和krb5.这两个都在环境中设置. krb5.ini(并设置为envt变量KRB5_CONFIG)

I'm quite new to Kafka so please bear with me. Here is my set up. I have kafka hosted on a unix box. Clustered. and in a domain say B. client is on windows. and am trying to connect to kafka hosted on B using a domain A. I have the keytab. and krb5. both these are set up in the envt. krb5.ini(and is set to envt variable KRB5_CONFIG)

 [logging]
 default = CONSOLE
 admin_server = CONSOLE
 kdc = CONSOLE

[libdefaults]
 renew_lifetime = 7d
 clockskew = 324000
 forwardable = true
 proxiable = true
 renewable = true
 default_realm = some.something.COM
  dns_lookup_realm = true
  dns_lookup_kdc = false
 default_tgs_enctypes = somethingelse
 default_tkt_enctypes = somethingelse

 [appdefaults]
   renewable = true

  [realms]
   some.something.COM = {
     kdc = some.something.COM
     admin_server = some.something.COM
 }

我还设置了Jaas.config(以我的情况为Kafka.client.ini，并设置为envt变量KAFKA_CLIENT_KERBEROS_PARAMS)，以下是配置

I also have set up Jaas.config(Kafka.client.ini in my case and is set to envt variable KAFKA_CLIENT_KERBEROS_PARAMS) below is the config

   KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="sample.keytab"
   storeKey=true
   useTicketCache=true
   serviceName="kafka"
   principal="[e-mail]";

};

下载了apache kafka_2.12-0.10.2.1.tgz并正在执行此命令.

downloaded apache kafka_2.12-0.10.2.1.tgz and am executing this command.

kafka-console-producer.bat --broker-list <broker list> --topic <mytopic>    --security-protocol SASL_PLAINTEXT

无论我做了什么更改，我始终会遇到错误

no matter what i change i keep getting below error

安全协议不是公认的选项"

"security-protocol is not a recognised option"

有人可以帮我吗? 我还在producer.properties中的props下面添加了.但似乎没有任何改变.我不确定我缺少什么

can someone please help me in this? i also added below props in producer.properties. but nothing seems to change. I'm not sure what i'm missing

security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka

我什至尝试在kafka-console-producer.bat中设置此属性，但是没有运气

I even tried setting this property in kafka-console-producer.bat but with no luck

set KAFKA_CLIENT_KERBEROS_PARAMS=- Djava.security.auth.login.config=..\..\config\kafka_Connection.ini

期待您的投入.非常感谢(我目前无法控制kafka服务器，也无法解释为什么它托管在域B中)

looking forward for your inputs. Many thanks (i've no control as of now on kafka server nor i will be able to explain why its hosted on domain B)

推荐答案

免责声明:我对Kafka不太熟悉，并且该错误消息也不能明确暗示Kerberos问题.
但是鉴于这是跨领域的情况，您迟早会 遇到Kerberos障碍...

Disclaimer: I'm not too familiar with Kafka, and that error message does not clearly hint at a Kerberos problem.
But given that this is a cross-realm situation, you will probably hit a Kerberos snag sooner or later...

从Kerberos MIT文档中的krb5.conf部分 [capaths]

From Kerberos MIT documentation about krb5.conf section [capaths]

为了执行 direct (非分层)跨领域 身份验证，需要配置来确定 领域之间的身份验证路径.

In order to perform direct (non-hierarchical) cross-realm authentication, configuration is needed to determine the authentication paths between realms.

客户端将使用此部分查找之间的身份验证路径 它的领域和服务器的领域.

A client will use this section to find the authentication path between its realm and the realm of the server.

换句话说，对于主体[e-mail]，您将获得Kerberos TGT(票证授予票)，但对于kafka/[e-mail]，则需要Kerberos服务票证.每个领域都有其自己的KDC服务器.您的Kerberos客户端(在这种情况下为Java实现)必须具有从一个域跳"到另一个域

In other words, you get a Kerberos TGT (ticket-granting-ticket) for principal [e-mail] but need a Kerberos service ticket for kafka/[e-mail]. Each realm has its own KDC servers. Your Kerberos client (the Java implementation in this case) must have a way to "hop" from one domain to the others

---

方案1 >>这两个领域都是相互信任的兄弟" AD域，并且它们使用默认的等级关系-表示在USERS到SERVERS的路径中有一个名为CORP.DMN的父亲" AD域.

---

Scenario 1 >> both realms are "brother" AD domains with mutual trust, and they use the default hierarchical relationship -- meaning that there is a "father" AD domain named CORP.DMN that is in the path from USERS to SERVERS.

您的krb5.conf应该看起来像这样...

Your krb5.conf should look like this...

[libdefaults]
default_realm = USERS.CORP.DMN
kdc_timeout   = 3000
...

...

[realms]
USERS.CORP.DMN = {
  kdc = roundrobin.siteA.users.corp.dmn
  kdc = roundrobin.bcp.users.corp.dmn
}
SERVERS.CORP.DMN = {
  kdc = dc1.servers.corp.dmn
  kdc = dc2.servers.corp.dmn
  kdc = roundrobin.bcp.servers.corp.dmn
}
CORP.DMN = {
  kdc = roundrobin.corp.dmn
  kdc = roundrobin.bcp.corp.dmn
}

...假设您在每个域中都有多个AD域控制器，有时在DNS别名后面进行循环分配，以及在单独的站点上为BCP/DRP设置另一组DC.可能比这更简单:-)

...assuming you have multiple AD Domain Controllers in each domain, sometimes behind DNS aliases doing round-robin assignment, plus another set of DC on a separate site for BCP/DRP. It could be more simple than that :-)

---

方案2 >>已启用信任，但关系不使用默认的分层路径.

---

Scenario 2 >> there is trust enabled but the relationship does not use the default, hierarchical path.

在这种情况下，您必须在[capaths]部分中明确定义该路径"，如Kerberos文档中所述.

In that case you must define explicitly that "path" in a [capaths] section, as explained in the Kerberos documentation.

---

场景3 >>域之间没有信任.你被搞砸了.

---

Scenario 3 >> there is no trust between realms. You are screwed.

或者，您必须获得可以与Kafka经纪人在同一域上进行身份验证的其他用户，例如[e-mail].
也许可以使用声明default_realm = SERVERS.CORP.DMN 的特定krb5.conf(例如，我已经看到Windows上某些JDK版本的怪异行为)

Or rather, you must obtain a different user that can authenticate on the same domain as the Kafka broker, e.g. [e-mail].
And maybe use a specific krb5.conf that states default_realm = SERVERS.CORP.DMN (I've seen weird behaviors of some JDK versions on Windows, for example)

---

底线:您必须要求广告管理员的帮助.也许他们不熟悉原始Kerberos conf，但是他们将了解信任关系和路径".此时，只需遵循正确的krb5.conf语法即可.

或者，也许conf已经由Linux管理员完成了；因此，您应该提供一个标准krb5.conf的示例，以检查其中是否存在跨域内容.

Or, maybe, that conf has already been done by the Linux administrators; so you should require an example of their standard krb5.conf to check whether there is cross-domain stuff in there.

当然，您应该在Kafka生产者中启用 Kerberos调试跟踪:

And of course you should enable Kerberos debug traces in your Kafka producer:

-Dsun.security.krb5.debug=true
-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext

仅作记录，但在这里没有用...在HTTP(SPNego)上使用Keberos时，还有一个附加标志-Dsun.security.spnego.debug=true

Just for the record, but not useful here... when using Keberos over HTTP (SPNego) there's an additional flag-Dsun.security.spnego.debug=true

这篇关于使用Kerberos从Windows连接到Unix上的Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！