可以在启动docker时通过传递的env变量来配置kafka连接器吗?还是卷曲是唯一的方法? [英] Can the kafka connectors be configured via env variables passed when launching docker? Or curl is the only way?
问题描述
这是我们用来托管与插件进行docker-connect的docker镜像
This is the docker image we use to host docker-connect with the plugins
FROM confluentinc/cp-kafka-connect:5.3.1
ENV CONNECT_PLUGIN_PATH=/usr/share/java
# JDBC-MariaDB
RUN wget -nv -P /usr/share/java/kafka-connect-jdbc/ https://downloads.mariadb.com/Connectors/java/connector-java-2.4.4/mariadb-java-client-2.4.4.jar
# SNMP Source
RUN wget -nv -P /tmp/ https://github.com/name/kafka-connect-snmp/releases/download/0.0.1.11/kafka-connect-snmp-0.0.1.11.tar.gz
RUN mkdir /tmp/kafka-connect-snmp && tar -xf /tmp/kafka-connect-snmp-0.0.1.11.tar.gz -C /tmp/kafka-connect-snmp/
RUN mv /tmp/kafka-connect-snmp/usr/share/kafka-connect/kafka-connect-snmp /usr/share/java/
我通过docker-compose运行此docker,然后指定了一些在此处定义的常见环境变量
I run this docker via docker-compose and then I have specified some common env variables defined here https://docs.confluent.io/current/installation/docker/config-reference.html#kafka-connect-configuration
但是我也想从env变量中指定与连接器相关的配置,例如我已经完成的
But I also would like to specify connector related config from the env variable also, example I have done this
- CONNECT_NAME=snmp-connector
- CONNECT_CONNECTOR_CLASS=com.github.jcustenborder.kafka.connect.snmp.SnmpTrapSourceConnector
- CONNECT_TOPIC=fm_snmp
我正在尝试做的事情,而不是打电话给
What I am trying to do it, instead of calling
curl -X POST -H "Content-Type: application/json" --data '{"name":"","config":{"connector.class":"com.github.jcustenborder.kafka.connect.snmp.SnmpTrapSourceConnector","topic":"fm_snmp"}}' http://localhost:8083/connectors
我只想通过env变量来指定它,但是!不幸的是,它无法正常工作.因此,当我尝试查看活动连接器curl -localhost:8083/connectors/的列表时,则没有在其中列出它.
I want to just specify it via env variables, BUT!! unfortunately its not working. So when I try seeing list of active connectors curl -localhost:8083/connectors/ , then I dont see it listed there.
所以最后,我的问题是我可以通过env变量配置它还是只能通过curl进行配置?
So finally, my question can I configure it via env variables or only curl is the way?
推荐答案
您不能将其作为环境变量传递,但可以通过传递自定义命令将其指定为Docker启动的一部分.这是使用Docker Compose进行操作的示例.如果您自己调用docker run
,则需要将其重新整理为适当的结构:
You can't pass it as environment variables, but you can specify it as part of your Docker startup by passing in a custom command. Here's an example of doing it with Docker Compose. If you're calling docker run
itself you'd need to rework this into an appropriate structure:
kafka-connect:
image: confluentinc/cp-kafka-connect:5.3.1
environment:
CONNECT_REST_PORT: 18083
CONNECT_REST_ADVERTISED_HOST_NAME: "kafka-connect"
[…]
volumes:
- $PWD/scripts:/scripts
command:
- bash
- -c
- |
/etc/confluent/docker/run &
echo "Waiting for Kafka Connect to start listening on kafka-connect ⏳"
while [ $$(curl -s -o /dev/null -w %{http_code} http://kafka-connect:8083/connectors) -eq 000 ] ; do
echo -e $$(date) " Kafka Connect listener HTTP state: " $$(curl -s -o /dev/null -w %{http_code} http://kafka-connect:8083/connectors) " (waiting for 200)"
sleep 5
done
nc -vz kafka-connect 8083
echo -e "\n--\n+> Creating Kafka Connect Elasticsearch sink"
/scripts/create-es-sink.sh
sleep infinity
This calls a connector script, but if you want to embed it directly you can do it like this.
这篇关于可以在启动docker时通过传递的env变量来配置kafka连接器吗?还是卷曲是唯一的方法?的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!