如何在 Kubernetes 中外部公开 StatefulSet 的无头服务 [英] How to expose a headless service for a StatefulSet externally in Kubernetes

问题描述

使用 kubernetes-kafka 作为 minikube 的起点.

Using kubernetes-kafka as a starting point with minikube.

这使用 StatefulSet 和 无头服务来提供服务在集群内发现.

This uses a StatefulSet and a headless service for service discovery within the cluster.

目标是在外部公开各个 Kafka Brokers，内部处理为:

The goal is to expose the individual Kafka Brokers externally which are internally addressed as:

kafka-0.broker.kafka.svc.cluster.local:9092
kafka-1.broker.kafka.svc.cluster.local:9092 
kafka-2.broker.kafka.svc.cluster.local:9092

限制是此外部服务能够专门解决代理.

The constraint is that this external service be able to address the brokers specifically.

解决此问题的正确(或一种可能)方法是什么?是否可以根据 kafka-x.broker.kafka.svc.cluster.local:9092 公开外部服务?

Whats the right (or one possible) way of going about this? Is it possible to expose a external service per kafka-x.broker.kafka.svc.cluster.local:9092?

推荐答案

到目前为止的解决方案对我来说还不够令人满意，所以我将发布我自己的答案.我的目标:

Solutions so far weren't quite satisfying enough for myself, so I'm going to post an answer of my own. My goals:

1. Pod 仍应尽可能通过 StatefulSet 进行动态管理.
2. 为生产者/消费者客户端的每个 Pod(即 Kafka Broker)创建一个外部服务并避免负载平衡.
3. 创建一个内部无头服务，以便每个 Broker 可以相互通信.

从 Yolean/kubernetes-kafka 开始，唯一缺少的就是对外暴露服务和这样做的两个挑战.

Starting with Yolean/kubernetes-kafka, the only thing missing is exposing the service externally and two challenges in doing so.

1. 为每个 Broker pod 生成唯一标签，以便我们可以为每个 Broker pod 创建外部服务.
2. 告诉 Broker 使用内部 Service 相互通信，同时配置 Kafka 以告诉生产者/消费者通过外部 Service 进行通信.

每个 pod 标签和外部服务:

要为每个 Pod 生成标签，这个问题真的很有帮助.使用它作为指导，我们将以下行添加到 10broker-config.yml init.sh 属性:

To generate labels per pod, this issue was really helpful. Using it as a guide, we add the following line to the 10broker-config.yml init.sh property with:

kubectl label pods ${HOSTNAME} kafka-set-component=${HOSTNAME}

我们保留现有的无头服务，但我们也使用标签为每个 pod 生成一个外部服务(我将它们添加到 20dns.yml):

We keep the existing headless service, but we also generate an external Service per pod using the label (I added them to 20dns.yml):

apiVersion: v1
kind: Service
metadata:
  name: broker-0
   namespace: kafka
spec:
  type: NodePort
  ports:
  - port: 9093
    nodePort: 30093
selector:
  kafka-set-component: kafka-0

使用内部/外部侦听器配置 Kafka

我发现这个问题在尝试了解如何配置 Kafka 时非常有用.

I found this issue incredibly useful in trying to understand how to configure Kafka.

这再次需要更新 10broker-config.yml 包含以下内容:

This again requires updating the init.sh and server.properties properties in 10broker-config.yml with the following:

将以下内容添加到 server.properties 以更新安全协议(当前使用 PLAINTEXT):

Add the following to the server.properties to update the security protocols (currently using PLAINTEXT):

listener.security.protocol.map=INTERNAL_PLAINTEXT:PLAINTEXT,EXTERNAL_PLAINTEXT:PLAINTEXT
inter.broker.listener.name=INTERNAL_PLAINTEXT

动态确定init.sh中每个Pod的外部IP和外部端口:

Dynamically determine the external IP and for external port for each Pod in the init.sh:

EXTERNAL_LISTENER_IP=<your external addressable cluster ip>
EXTERNAL_LISTENER_PORT=$((30093 + ${HOSTNAME##*-}))

然后为 EXTERNAL_LISTENER 和 INTERNAL_LISTENER 配置 listeners 和 advertised.listeners IP(也在 init.sh 属性):

Then configure listeners and advertised.listeners IPs for EXTERNAL_LISTENER and INTERNAL_LISTENER (also in the init.sh property):

sed -i "s/#listeners=PLAINTEXT:\/\/:9092/listeners=INTERNAL_PLAINTEXT:\/\/0.0.0.0:9092,EXTERNAL_PLAINTEXT:\/\/0.0.0.0:9093/" /etc/kafka/server.properties
sed -i "s/#advertised.listeners=PLAINTEXT:\/\/your.host.name:9092/advertised.listeners=INTERNAL_PLAINTEXT:\/\/$HOSTNAME.broker.kafka.svc.cluster.local:9092,EXTERNAL_PLAINTEXT:\/\/$EXTERNAL_LISTENER_IP:$EXTERNAL_LISTENER_PORT/" /etc/kafka/server.properties

显然，这不是生产的完整解决方案(例如解决外部暴露的代理的安全问题)，我仍在完善我对如何让内部生产者/消费者也与代理进行通信的理解.

Obviously, this is not a full solution for production (for example addressing security for the externally exposed brokers) and I'm still refining my understanding of how to also let internal producer/consumers to also communicate with the brokers.

然而，到目前为止，这是我理解 Kubernetes 和 Kafka 的最佳方法.

However, so far this is the best approach for my understanding of Kubernetes and Kafka.

这篇关于如何在 Kubernetes 中外部公开 StatefulSet 的无头服务的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！