是否可以将 celery 与 Kafka 集成 [英] Is it possible to integrate celery with Kafka

问题描述

我想在我的代码中使用 celery 引入多处理.但目前我的队列实现是在 Kafka 中.

I want to introduce multiprocessing in my code using celery. But currently my queue implementation is in Kafka.

目前 celery 网站只提到了这 4 个经纪人:http://docs.celeryproject.org/en/master/getting-started/brokers/index.html#broker-overview

Currently celery website mentions of only these 4 brokers: http://docs.celeryproject.org/en/master/getting-started/brokers/index.html#broker-overview

是否可以将 Celery 与 Kafka 集成类似于下面提到的 RabbitMQ:

Is it possible to integrate Celery with Kafka something similar to RabbitMQ mentioned below:

from celery import Celery
app = Celery('tasks', broker='pyamqp://guest@localhost//')
@app.task
def add(x, y):
    return x + y

推荐答案

有人讨论过让 kafka 作为 celery 的 broker，并且有部分实现.希望对你有帮助

There was discussion on having kafka as broker for celery, And there is a partial implementation of it. I hope this helps you

https://github.com/celery/kombu/issues/301

看起来它将于 10 月 31 日左右上市.

Looks like it will be available this Oct 31 or so.

https://github.com/celery/kombu/milestone/2

这篇关于是否可以将 celery 与 Kafka 集成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！