使用 Kerberos 从 Windows 连接到 Unix 上的 Kafka [英] Connect to Kafka on Unix from Windows with Kerberos

查看:91
本文介绍了使用 Kerberos 从 Windows 连接到 Unix 上的 Kafka的处理方法,对大家解决问题具有一定的参考价值,需要的朋友们下面随着小编来一起学习吧!

问题描述

我对卡夫卡很陌生,所以请多多包涵.这是我的设置.我在 unix 机器上托管了 kafka.集群.在域中说 B.客户端在 Windows 上.我正在尝试使用域 A 连接到托管在 B 上的 kafka.我有密钥表.和 krb5.这两个都是在envt中设置的.krb5.ini(并设置为envt变量KRB5_CONFIG)

I'm quite new to Kafka so please bear with me. Here is my set up. I have kafka hosted on a unix box. Clustered. and in a domain say B. client is on windows. and am trying to connect to kafka hosted on B using a domain A. I have the keytab. and krb5. both these are set up in the envt. krb5.ini(and is set to envt variable KRB5_CONFIG)

 [logging]
 default = CONSOLE
 admin_server = CONSOLE
 kdc = CONSOLE

[libdefaults]
 renew_lifetime = 7d
 clockskew = 324000
 forwardable = true
 proxiable = true
 renewable = true
 default_realm = some.something.COM
  dns_lookup_realm = true
  dns_lookup_kdc = false
 default_tgs_enctypes = somethingelse
 default_tkt_enctypes = somethingelse

 [appdefaults]
   renewable = true

  [realms]
   some.something.COM = {
     kdc = some.something.COM
     admin_server = some.something.COM
 }

我还设置了 Jaas.config(在我的情况下为 Kafka.client.ini 并设置为 envt 变量 KAFKA_CLIENT_KERBEROS_PARAMS)下面是配置

I also have set up Jaas.config(Kafka.client.ini in my case and is set to envt variable KAFKA_CLIENT_KERBEROS_PARAMS) below is the config

   KafkaClient {
    com.sun.security.auth.module.Krb5LoginModule required
   useKeyTab=true
   keyTab="sample.keytab"
   storeKey=true
   useTicketCache=true
   serviceName="kafka"
   principal="svcacc@some.something.COM";

};

已下载 apache kafka_2.12-0.10.2.1.tgz 并正在执行此命令.

downloaded apache kafka_2.12-0.10.2.1.tgz and am executing this command.

kafka-console-producer.bat --broker-list <broker list> --topic <mytopic>    --security-protocol SASL_PLAINTEXT

无论我改变什么,我都会收到以下错误

no matter what i change i keep getting below error

安全协议不是公认的选项"

"security-protocol is not a recognised option"

有人可以帮我吗?我还在 producer.properties 中添加了以下道具.但似乎什么都没有改变.我不确定我错过了什么

can someone please help me in this? i also added below props in producer.properties. but nothing seems to change. I'm not sure what i'm missing

security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka

我什至尝试在 kafka-console-producer.bat 中设置此属性,但没有成功

I even tried setting this property in kafka-console-producer.bat but with no luck

set KAFKA_CLIENT_KERBEROS_PARAMS=- Djava.security.auth.login.config=..\..\config\kafka_Connection.ini

期待您的意见.非常感谢(我现在无法控制 kafka 服务器,也无法解释为什么它托管在域 B 上)

looking forward for your inputs. Many thanks (i've no control as of now on kafka server nor i will be able to explain why its hosted on domain B)

推荐答案

免责声明:我对 Kafka 不太熟悉,该错误消息并未明确暗示 Kerberos 问题.
但鉴于这是一个跨领域的情况,您迟早会可能遇到 Kerberos 障碍...

Disclaimer: I'm not too familiar with Kafka, and that error message does not clearly hint at a Kerberos problem.
But given that this is a cross-realm situation, you will probably hit a Kerberos snag sooner or later...

来自关于 krb5.conf 部分的 Kerberos MIT 文档[capaths]

From Kerberos MIT documentation about krb5.conf section [capaths]

为了执行直接(非分层)跨领域身份验证,需要配置来确定领域之间的身份验证路径.

In order to perform direct (non-hierarchical) cross-realm authentication, configuration is needed to determine the authentication paths between realms.

客户端将使用此部分来查找之间的身份验证路径它的领域和服务器的领域.

A client will use this section to find the authentication path between its realm and the realm of the server.

换句话说,您获得了主体 wtf@USERS.CORP.DMN 的 Kerberos TGT(ticket-granting-ticket),但需要 kafka/brokerhost.some 的 Kerberos 服务票证.where@SERVERS.CORP.DMN.每个领域都有自己的 KDC 服务器.您的 Kerberos 客户端(在本例中为 Java 实现) 必须有一种方法可以从一个域跳转"到其他域

In other words, you get a Kerberos TGT (ticket-granting-ticket) for principal wtf@USERS.CORP.DMN but need a Kerberos service ticket for kafka/brokerhost.some.where@SERVERS.CORP.DMN. Each realm has its own KDC servers. Your Kerberos client (the Java implementation in this case) must have a way to "hop" from one domain to the others


场景 1 >> 两个领域都是相互信任的兄弟"AD 域,它们使用默认的层级关系-- 意味着在从 USERSSERVERS 的路径中有一个名为 CORP.DMN 的父"AD 域.


Scenario 1 >> both realms are "brother" AD domains with mutual trust, and they use the default hierarchical relationship -- meaning that there is a "father" AD domain named CORP.DMN that is in the path from USERS to SERVERS.

你的 krb5.conf 应该是这样的...

Your krb5.conf should look like this...

[libdefaults]
default_realm = USERS.CORP.DMN
kdc_timeout   = 3000
...

...

[realms]
USERS.CORP.DMN = {
  kdc = roundrobin.siteA.users.corp.dmn
  kdc = roundrobin.bcp.users.corp.dmn
}
SERVERS.CORP.DMN = {
  kdc = dc1.servers.corp.dmn
  kdc = dc2.servers.corp.dmn
  kdc = roundrobin.bcp.servers.corp.dmn
}
CORP.DMN = {
  kdc = roundrobin.corp.dmn
  kdc = roundrobin.bcp.corp.dmn
}

...假设您在每个域中有多个 AD 域控制器,有时在 DNS 别名后面进行循环分配,另外还有一组位于单独站点上的 DC 用于 BCP/DRP.可能比这更简单:-)

...assuming you have multiple AD Domain Controllers in each domain, sometimes behind DNS aliases doing round-robin assignment, plus another set of DC on a separate site for BCP/DRP. It could be more simple than that :-)


场景 2 >> 启用了信任,但关系不使用默认的分层路径.


Scenario 2 >> there is trust enabled but the relationship does not use the default, hierarchical path.

在这种情况下,您必须在 [capaths] 部分中明确定义该路径",如 Kerberos 文档中所述.

In that case you must define explicitly that "path" in a [capaths] section, as explained in the Kerberos documentation.


场景 3 >> 领域之间不存在信任.你完蛋了.


Scenario 3 >> there is no trust between realms. You are screwed.

或者更确切地说,您必须获得可以在与 Kafka 代理相同的域上进行身份验证的不同用户,例如xyz@SERVERS.CORP.DMN.
并且可能使用特定的 krb5.conf 声明 default_realm = SERVERS.CORP.DMN (例如,我在 Windows 上看到了某些 JDK 版本的奇怪行为)

Or rather, you must obtain a different user that can authenticate on the same domain as the Kafka broker, e.g. xyz@SERVERS.CORP.DMN.
And maybe use a specific krb5.conf that states default_realm = SERVERS.CORP.DMN (I've seen weird behaviors of some JDK versions on Windows, for example)


底线:您必须寻求 AD 管理员的帮助.也许他们不熟悉原始 Kerberos conf,但他们会了解信任和路径";在这一点上,只需遵循正确的 krb5.conf 语法即可.

或者,也许那个 conf 已经由 Linux 管理员完成了;所以你应该需要一个他们的标准 krb5.conf 的例子来检查那里是否有跨域的东西.

Or, maybe, that conf has already been done by the Linux administrators; so you should require an example of their standard krb5.conf to check whether there is cross-domain stuff in there.

当然,您应该在 Kafka 生产者中启用 Kerberos 调试跟踪:

And of course you should enable Kerberos debug traces in your Kafka producer:

-Dsun.security.krb5.debug=true
-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext

只是为了记录,但在这里没有用...当通过 HTTP (SPNEgo) 使用 Keberos 时,有一个额外的标志-Dsun.security.spnego.debug=true

Just for the record, but not useful here... when using Keberos over HTTP (SPNego) there's an additional flag-Dsun.security.spnego.debug=true

这篇关于使用 Kerberos 从 Windows 连接到 Unix 上的 Kafka的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!

查看全文
登录 关闭
扫码关注1秒登录
发送“验证码”获取 | 15天全站免登陆