kafka 连接器 HTTP/API 源 [英] kafka connector HTTP/API source

问题描述

我实际上知道如何从任何数据源捕获数据，例如特定的 API(例如 HTTP GET 请求)并将它们摄取到特定的 kafka 连接器中.

<代码>{"name": "localfileSource",配置":{"connector.class": "FileStreamSourceConnector","tasks.max": "1","file": "test.txt",主题":连接源"}}

我需要类似的东西(FileStreamSourceConnector)，可以与 API 源一起使用.

解决方案

提出问题可能有点晚了，但我有类似的需求，我决定实现自己的连接器.

连接器的特殊性在于它实现了某种变更数据捕获

如果它可以帮助其他人，你可以在这里找到它:castorm/kafka-connect-http

I am actually aware on how to capture data from any data source, such as a specific API (e.g HTTP GET request) and ingest them in specific kafka connector.

{
    "name": "localfileSource", 
    "config": {
        "connector.class": "FileStreamSourceConnector",
        "tasks.max": "1",
        "file": "test.txt",
        "topic": "connectSource"
    }
}

I would need something similar to this(FileStreamSourceConnector), that can be used with API sources.

解决方案

It's probably a bit late for the question asker, but I had a similar need and I decided to implement my own connector.

The particularity of the connector is that it implements some sort of Change Data Capture

In case it can help anyone else, you can find it here: castorm/kafka-connect-http

这篇关于kafka 连接器 HTTP/API 源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！