阅读后如何删除kafka消息 [英] how to delete kafka message after reading

问题描述

我正在使用下面的代码从某个主题中读取消息.阅读邮件后如何删除邮件?

I am using the below code to read messages from a topic. How do i delete a message after it is read?

from kafka import KafkaConsumer


    consumer = KafkaConsumer('my-topic',
                             group_id='my-group',
                             bootstrap_servers=['localhost:9092'])
    for message in consumer:
        # message value and key are raw bytes -- decode if necessary!
        # e.g., for unicode: `message.value.decode('utf-8')`
        print ("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition,
                                              message.offset, message.key,
                                              message.value))

推荐答案

无法从kafka删除特定的消息-kafka根本不打算这样做.删除消息的唯一方法是将kafka的 config/server.properties 中的 log.retention.hours 设置为您喜欢的值.默认值为168-表示168小时后不保留邮件.

There is no way to delete a specific message from kafka - kafka simply is not designed to do that. The only way to delete messages is by setting log.retention.hours in kafka's config/server.properties to a value of your liking. The default is 168 - meaning that messages are not kept after 168 hours.

如果您正在寻找一种从特定偏移量读取消息的方法-即不是每次都从头开始读取，请在此处

If you instead are looking for a way to read messages from a specific offset - i.e. not read from the beginning every time, look here http://kafka-python.readthedocs.org/en/master/apidoc/KafkaConsumer.html
commit() - committing read offsets to kafka
seek_to_end() - fast forward to consuming only newly arriving messages
seek() - moving to a given offset (presumably stored somewhere else than in kafka)

这篇关于阅读后如何删除kafka消息的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！