手动安装Kafka连接器 [英] Install Kafka connector manually

问题描述

我正在尝试安装twitter源连接器，并找到以下链接链接1 和 link2 。第一个显示如何安装twitter源连接器，第二个显示一般如何手动安装连接器（始终为Confluent Platform）。

I'm trying install twitter source connector and found the following links link1 and link2. The first one shows how to install twitter source connector and the second generally how to install manually a connector (always Confluent Platform).

但是我对整个过程感到困惑。我遵循了此处，到目前为止，一切正常。但是，由于twitter源连接器不在连接器中心的列表中，因此需要手动安装。根据link1，我在某个地方克隆了一个git repo-问题：在哪里克隆它？在docker-compose.yml中，有一堆服务。特别是，在我旋转Yaml显示后， docker-compose ps

However I'm confused from the entire procedure. I followed the steps in here, and so far it's working fine. However since the twitter source connector is not in the list of the Connector Hub it needs to be installed manually. According to link1, I clone a git repo somewhere - Question: where do I clone it? In the docker-compose.yml there are a bunch of services. In particular, a docker-compose ps after I spin up the yaml shows

       Name                    Command                  State                         Ports
  ------------------------------------------------------------------------------------------------------------

   broker            /etc/confluent/docker/run        Up             0.0.0.0:9092->9092/tcp
   connect           /etc/confluent/docker/run        Up             0.0.0.0:8083->8083/tcp, 9092/tcp
   control-center    /etc/confluent/docker/run        Up             0.0.0.0:9021->9021/tcp
   ksql-datagen      bash -c echo Waiting for K ...   Up
   ksqldb-cli        /bin/sh                          Up
   ksqldb-server     /etc/confluent/docker/run        Up (healthy)   0.0.0.0:8088->8088/tcp
   rest-proxy        /etc/confluent/docker/run        Up             0.0.0.0:8082->8082/tcp
   schema-registry   /etc/confluent/docker/run        Up             0.0.0.0:8081->8081/tcp
   zookeeper         /etc/confluent/docker/run        Up             0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp

我不知道我应该对link1进行上述操作。是在动物园管理员，经纪人还是在连接中？欢迎任何帮助，如果有人有时间更全面地围绕link1的步骤写出几句话，请放心。欢呼声

I don't understand to which one of the above I should carry out the instructions of link1. Is it in the zookeeper, the broker or in the connect? Any help is welcomed, please if someone has time to write out a few words around the steps in link1 more thoroughly, please feel free. Cheers

推荐答案

连接器安装在Kafka Connect工作程序中。必须在启动工作程序之前安装它们。

Connectors are installed in Kafka Connect workers. They must be installed before the worker is started.

该博客文章有一个较新的版本，在这里： https://www.confluent.io/blog/stream-processing-twitter-data-with-ksqldb 。与此相关的是一个新的演示，您可以在这里找到： https： //github.com/confluentinc/demo-scene/tree/master/ksqldb-twitter

There is a newer version of that blog article, here: https://www.confluent.io/blog/stream-processing-twitter-data-with-ksqldb. Associated with that is a new demo, which you can find here: https://github.com/confluentinc/demo-scene/tree/master/ksqldb-twitter

作为演示的一部分，它安装了twitter连接器< a href = https://github.com/confluentinc/demo-scene/blob/master/ksqldb-twitter/docker-compose.yml#L108 rel = nofollow noreferrer>在运行时。它通过覆盖容器映像的默认命令来执行此操作，而是在运行连接器之前运行连接器安装：

As part of the demo, it installs the twitter connector at runtime. It does this by overriding the default command of the container image and instead runs the connector installation before running the connect worker:

    command: 
      - bash 
      - -c 
      - |
        echo "Installing connector plugins"
        confluent-hub install --no-prompt jcustenborder/kafka-connect-twitter:0.3.33
        #
        echo "Launching Kafka Connect worker"
        /etc/confluent/docker/run & 
        #
        sleep infinity

这篇关于手动安装Kafka连接器的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！