如何从外部连接到 Kubernetes 上的 Kafka [英] How to connect to Kafka on Kubernetes externally
问题描述
我使用 Yolean/kubernetes-kafka 在本地 Docker (gcp & minikube) 上成功地将 Kafka 部署到 Kubernetes &Helm 图表
并使用此 python 脚本从集群内成功测试主题生产:
and tested topic production successfully from within the cluster using this python script:
#!/usr/bin/env python
from kafka import KafkaConsumer, KafkaProducer
KAFKA_TOPIC = 'demo'
# KAFKA_BROKERS = 'localhost:32400' # see step 1
# from inside the cluster in a different namespace
# KAFKA_BROKERS = 'bootstrap.kafka.svc.cluster.local:9092'
KAFKA_BROKERS = 'kafka.kafka.svc.cluster.local:9092'
print('KAFKA_BROKERS: ' + KAFKA_BROKERS)
producer = KafkaProducer(bootstrap_servers=KAFKA_BROKERS)
messages = [b'hello kafka', b'Falanga', b'3 test messages']
for m in messages:
print(f"sending: {m}")
producer.send(KAFKA_TOPIC, m)
producer.flush()
掌舵时，我使用此选项启用外部使用:
On helm I used this option to enable external use:
helm install --name kafka --set external.enabled=true --namespace kafka incubator/kafka
以及我使用的原始存储库:
and on the original repo I used:
kubectl apply -f ./outside-0.yml
生成的服务有端点和节点端口，但脚本在集群外部不起作用.
The resulting services have endpoints and node ports but the script doesn't work from outside the cluster.
这里是原始服务(branch master)
here is the original service (branch master)
➜ ~ kubectl describe svc outside-0 --namespace kafka
Name: outside-0
Namespace: kafka
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied- configuration={"apiVersion":"v1","kind":"Service","metadata": {"annotations":{},"name":"outside-0","namespace":"kafka"},"spec":{"ports": [{"nodePort":32400,"port":3240...
Selector: app=kafka,kafka-broker-id=0
Type: NodePort
IP: 10.99.171.133
LoadBalancer Ingress: localhost
Port: <unset> 32400/TCP
TargetPort: 9094/TCP
NodePort: <unset> 32400/TCP
Endpoints: 10.1.3.63:9094
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
这里是 helm 服务描述:
here is the helm service description:
Name: kafka-0-external
Namespace: kafka
Labels: app=kafka
chart=kafka-0.9.2
heritage=Tiller
pod=kafka-0
release=kafka
Annotations: dns.alpha.kubernetes.io/internal=kafka.cluster.local
external- dns.alpha.kubernetes.io/hostname=kafka.cluster.local
Selector: app=kafka,pod=kafka-0,release=kafka
Type: NodePort
IP: 10.103.70.223
LoadBalancer Ingress: localhost
Port: external-broker 19092/TCP
TargetPort: 31090/TCP
NodePort: external-broker 31090/TCP
Endpoints: 10.1.2.231:31090
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
本地docker节点没有externalIP字段:
The local docker node does not have an externalIP field:
kubectl describe node docker-for-desktop | grep IP
InternalIP: 192.168.65.3
我按照外面的说明Readme 即
- 将 hostPort 添加到 50kafka statefullset 9094 端口
- 在 10broker-config 中添加节点端口发现
&发现本地docker节点没有externalIP字段
& discovered that the local docker node has no externalIP field
如何从 docker 上的集群外部连接到 kafka?这是否适用于 GKE 或其他部署?
How can I connect to kafka from outside the cluster on docker? Does this work on GKE or other deployments?
推荐答案
该服务正在将 pod 暴露给内部 Kubernetes 网络.为了向互联网公开服务(公开 pod)，您需要设置一个指向该服务的 Ingress.
The service is exposing the pod to the internal Kubernetes network. In order to expose the service (which exposes the pod) to the internet, you need to set up an Ingress that points to the service.
Ingresses 基本上相当于 Kubernetes 的 Apache/Nginx.您可以通过以下 URL 了解如何操作:
Ingresses are basically the equivalent of Apache/Nginx for Kubernetes. You can read up on how to do it at the following URL:
https://kubernetes.io/docs/concepts/services-networking/入口/
或者，您可以通过将 service type 定义为 NodePort 并将您的特定端口分配给它来在节点网络上公开 pod.它应该类似于以下内容:
Alternatively, you can expose a pod on the node network by defining the service type as a NodePort and assigning your specific port to it. It should be something like the following:
<代码>api版本:v1种类:服务元数据:名称: nginx标签:名称: nginx规格:类型:节点端口端口:- 端口:80节点端口:31090名称:http
这篇关于如何从外部连接到 Kubernetes 上的 Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
