kafka连接器HTTP/API源 [英] kafka connector HTTP/API source

问题描述

我实际上知道如何从任何数据源(例如特定的API(例如HTTP GET请求))捕获数据并将其提取到特定的kafka连接器中.

{
    "name": "localfileSource", 
    "config": {
        "connector.class": "FileStreamSourceConnector",
        "tasks.max": "1",
        "file": "test.txt",
        "topic": "connectSource"
    }
}

我需要类似于this(FileStreamSourceConnector)的东西，并且可以与API源一起使用.

解决方案

对于提问者来说可能有点晚了，但是我有类似的需求，所以我决定实现自己的连接器.

连接器的特殊之处在于它实现了某种更改数据捕获

万一它可以帮助其他人，您可以在这里找到它: castorm/kafka-connect -http

I am actually aware on how to capture data from any data source, such as a specific API (e.g HTTP GET request) and ingest them in specific kafka connector.

{
    "name": "localfileSource", 
    "config": {
        "connector.class": "FileStreamSourceConnector",
        "tasks.max": "1",
        "file": "test.txt",
        "topic": "connectSource"
    }
}

I would need something similar to this(FileStreamSourceConnector), that can be used with API sources.

解决方案

It's probably a bit late for the question asker, but I had a similar need and I decided to implement my own connector.

The particularity of the connector is that it implements some sort of Change Data Capture

In case it can help anyone else, you can find it here: castorm/kafka-connect-http

这篇关于kafka连接器HTTP/API源的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！