如何从外部连接到Kubernetes上的Kafka [英] How to connect to Kafka on Kubernetes externally
问题描述
我使用 Yolean/kubernetes-kafka在本地Docker(gcp& minikube)上成功地将Kafka部署到Kubernetes. & 头盔图
并使用以下python脚本从集群内部成功测试了主题的产生:
and tested topic production successfully from within the cluster using this python script:
#!/usr/bin/env python
from kafka import KafkaConsumer, KafkaProducer
KAFKA_TOPIC = 'demo'
# KAFKA_BROKERS = 'localhost:32400' # see step 1
# from inside the cluster in a different namespace
# KAFKA_BROKERS = 'bootstrap.kafka.svc.cluster.local:9092'
KAFKA_BROKERS = 'kafka.kafka.svc.cluster.local:9092'
print('KAFKA_BROKERS: ' + KAFKA_BROKERS)
producer = KafkaProducer(bootstrap_servers=KAFKA_BROKERS)
messages = [b'hello kafka', b'Falanga', b'3 test messages']
for m in messages:
print(f"sending: {m}")
producer.send(KAFKA_TOPIC, m)
producer.flush()
在头盔上，我使用了此选项以启用外部使用:
On helm I used this option to enable external use:
helm install --name kafka --set external.enabled=true --namespace kafka incubator/kafka
以及我使用的原始存储库上
and on the original repo I used:
kubectl apply -f ./outside-0.yml
生成的服务具有端点和节点端口，但是该脚本无法在群集外部运行.
The resulting services have endpoints and node ports but the script doesn't work from outside the cluster.
这是原始服务(分支机构主服务)
here is the original service (branch master)
➜ ~ kubectl describe svc outside-0 --namespace kafka
Name: outside-0
Namespace: kafka
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied- configuration={"apiVersion":"v1","kind":"Service","metadata": {"annotations":{},"name":"outside-0","namespace":"kafka"},"spec":{"ports": [{"nodePort":32400,"port":3240...
Selector: app=kafka,kafka-broker-id=0
Type: NodePort
IP: 10.99.171.133
LoadBalancer Ingress: localhost
Port: <unset> 32400/TCP
TargetPort: 9094/TCP
NodePort: <unset> 32400/TCP
Endpoints: 10.1.3.63:9094
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
这是舵机服务说明:
Name: kafka-0-external
Namespace: kafka
Labels: app=kafka
chart=kafka-0.9.2
heritage=Tiller
pod=kafka-0
release=kafka
Annotations: dns.alpha.kubernetes.io/internal=kafka.cluster.local
external- dns.alpha.kubernetes.io/hostname=kafka.cluster.local
Selector: app=kafka,pod=kafka-0,release=kafka
Type: NodePort
IP: 10.103.70.223
LoadBalancer Ingress: localhost
Port: external-broker 19092/TCP
TargetPort: 31090/TCP
NodePort: external-broker 31090/TCP
Endpoints: 10.1.2.231:31090
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
本地docker节点没有externalIP字段:
The local docker node does not have an externalIP field:
kubectl describe node docker-for-desktop | grep IP
InternalIP: 192.168.65.3
我按照外面的说明自述文件即
- 将hostPort添加到50kafka statefullset 9094端口
- 在10broker-config中添加节点端口发现
&发现本地docker节点没有外部IP字段
& discovered that the local docker node has no externalIP field
如何从docker上的集群外部连接到kafka? 这对GKE或其他部署有效吗?
How can I connect to kafka from outside the cluster on docker? Does this work on GKE or other deployments?
推荐答案
该服务会将Pod暴露给内部Kubernetes网络.为了将服务(将pod暴露)公开给互联网，您需要设置一个指向该服务的Ingress.
The service is exposing the pod to the internal Kubernetes network. In order to expose the service (which exposes the pod) to the internet, you need to set up an Ingress that points to the service.
对于Kubernetes，入口基本上等同于Apache/Nginx.您可以通过以下URL阅读有关操作方法的信息:
Ingresses are basically the equivalent of Apache/Nginx for Kubernetes. You can read up on how to do it at the following URL:
https://kubernetes.io/docs/concepts/services-networking/入口/
或者，您可以通过将service type定义为NodePort并为其分配特定的端口，从而在节点网络上公开容器.它应该类似于以下内容:
Alternatively, you can expose a pod on the node network by defining the service type as a NodePort and assigning your specific port to it. It should be something like the following:
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
name: nginx
spec:
type: NodePort
ports:
- port: 80
nodePort: 31090
name: http
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
name: nginx
spec:
type: NodePort
ports:
- port: 80
nodePort: 31090
name: http
这篇关于如何从外部连接到Kubernetes上的Kafka的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
