Kafka Streams-低级处理器API-RocksDB TimeToLive(TTL) [英] Kafka Streams - Low-Level Processor API - RocksDB TimeToLive(TTL)

问题描述

我正在尝试使用低级处理器API.我正在使用处理器API对传入的记录进行数据聚合，并将聚合的记录写入RocksDB.

I'm kind of experimenting with the low level processor API. I'm doing data aggregation on incoming records using the processor API and writing the aggregated records to RocksDB.

但是，我想保留添加到rocksdb中的记录，使其仅在24小时内处于活动状态. 24小时后，该记录应删除.这可以通过更改ttl设置来完成.但是，没有太多文档可以为您提供帮助.

However, I want to retain the records added in the rocksdb to be active only for 24hr period. After 24hr period the record should be deleted. This can be done by changing the ttl settings. However, there is not much documentation where I can get some help on this.

如何更改ttl值?我应该使用哪种Java API将ttl时间设置为24小时?当前的默认ttl设置时间是多少?

how do I change the ttl value? What java api should I use to set the ttl time to 24 hrs and whats the current default ttl settings time?

推荐答案

我认为当前尚未通过api或配置公开. 打开RocksDB时，RocksDBStore传递硬编码的TTL:

I believe this is not currently exposed via the api or configuration. RocksDBStore passes a hard-coded TTL when opening a RocksDB: https://github.com/apache/kafka/blob/trunk/streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBStore.java#L158 and the hardcoded value is simply TTL_SECONDS = TTL_NOT_USED (-1) (see line 79 in that same file).

目前在州立商店中有2张关于公开TTL支持的公开票:KAFKA-4212和KAFKA-4273:

There are currently 2 open ticket regarding exposing TTL support in the state stores: KAFKA-4212 and KAFKA-4273: https://issues.apache.org/jira/issues/?jql=project%20%3D%20KAFKA%20AND%20text%20~%20%22rocksdb%20ttl%22 I suggest you comment on one of them describing your use case to get them moving forward.

在此期间，如果您现在需要TTL功能，则状态存储是可插入的，并且RocksDBStore源易于使用，因此您可以将其派生并设置TTL值(或类似与KAFKA-4273相关的拉取请求)建议，从配置中获取它.)

In the interim, if you need the TTL functionality right now, state stores are pluggable, and the RocksDBStore sources readily available, so you can fork it and set your TTL value (or, like the pull request associated with KAFKA-4273 proposes, source it from the configs).

我知道这不是理想的选择，并衷心希望有人提出一个更满意的答案.

I know this is not ideal and sincerely hope someone comes up with a more satisfactory answer.

这篇关于Kafka Streams-低级处理器API-RocksDB TimeToLive(TTL)的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！