Kubernetes上的多经纪人Kafka如何设置KAFKA_ADVERTISED_HOST_NAME [英] Multi-broker Kafka on Kubernetes how to set KAFKA_ADVERTISED_HOST_NAME

问题描述

我当前拥有3个Kafka经纪人的Kafka部署文件如下:

My current Kafka deployment file with 3 Kafka brokers looks like this:

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: kafka
spec:
  selector:
    matchLabels:
      app: kafka
  serviceName: kafka-headless
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  podManagementPolicy: Parallel
  template:
    metadata:
      labels:
        app: kafka
    spec:
      containers:
      - name: kafka-instance
        image: wurstmeister/kafka
        ports:
        - containerPort: 9092
        env:
        - name: KAFKA_ADVERTISED_PORT
          value: "9092"
        - name: KAFKA_ADVERTISED_HOST_NAME
          valueFrom:
              fieldRef:
                fieldPath: metadata.name
        - name: KAFKA_ZOOKEEPER_CONNECT
          value: "zookeeper-0.zookeeper-headless.default.svc.cluster.local:2181,\
                  zookeeper-1.zookeeper-headless.default.svc.cluster.local:2181,\
                  zookeeper-2.zookeeper-headless.default.svc.cluster.local:2181"
        - name: BROKER_ID_COMMAND
          value: "hostname | awk -F '-' '{print $2}'"
        - name: KAFKA_CREATE_TOPICS
          value: hello:2:1
        volumeMounts:
        - name: data
          mountPath: /var/lib/kafka/data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 50Gi

这将创建3个Kafka代理作为有状态集，并使用带有FQDN(完全合格域名)的Kubedns服务连接到Zookeeper集群，例如:

This creates 3 Kafka brokers as a Stateful Set and connects to the Zookeeper cluster using the Kubedns service with FQDN (Fully Qualified Domain Names) such as:

zookeeper-0.zookeeper-headless.default.svc.cluster.local:2181

经纪人ID是根据广告连播名称生成的:

Broker IDs are generated based on the pod name:

- name: BROKER_ID_COMMAND
          value: "hostname | awk -F '-' '{print $2}'"

结果:

kafka-0 = 0
kafka-1 = 1
kafka-2 = 2

但是，为了对Kafka经纪人使用Kubedns名称，

However, In order to use the Kubedns names for the Kafka brokers:

kafka-0.kafka-headless.default.svc.cluster.local:9092
kafka-1.kafka-headless.default.svc.cluster.local:9092
kafka-2.kafka-headless.default.svc.cluster.local:9092

我需要能够根据pod的名称将KAFKA_ADVERTISED_HOST_NAME变量设置为上述FQDN值.

I need to be able to set the KAFKA_ADVERTISED_HOST_NAME variable to the above FQDN values based on the name of the pod.

目前，我已将变量设置为广告连播的名称:

Currently I have the variable set to the name of the pod:

- name: KAFKA_ADVERTISED_HOST_NAME
   valueFrom:
      fieldRef:
        fieldPath: metadata.name

结果:

KAFKA_ADVERTISED_HOST_NAME=kafka-0
KAFKA_ADVERTISED_HOST_NAME=kafka-1
KAFKA_ADVERTISED_HOST_NAME=kafka-2

但是以某种方式，我需要附加其余的DNS名称.

But somehow I would need to append the rest of the DNS name.

有没有一种方法可以直接设置DNS值?

Is there a way I could set the DNS value directly?

类似的东西:

- name: KAFKA_ADVERTISED_HOST_NAME
       valueFrom:
          fieldRef:
            fieldPath: kubedns.name

推荐答案

我设法通过pod定义内的命令字段解决了这个问题:

I managed to solve the problem with a command field inside the pod definition:

    command:
    - sh
    - -c
    - "export KAFKA_ADVERTISED_HOST_NAME=$(hostname).kafka-headless.default.svc.cluster.local &&
       start-kafka.sh"

这将运行一个shell命令，该命令根据hostname值导出公告的主机名环境变量.

This runs a shell command which exports the advertised hostname environment variable based on the hostname value.

这篇关于Kubernetes上的多经纪人Kafka如何设置KAFKA_ADVERTISED_HOST_NAME的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！