如何为redis流定义TTL? [英] How to define TTL for redis streams?

问题描述

我有两个微服务，我需要在它们之间实现可靠的通知.我想过使用 redis 流 -serviceA 将向 serviceB 发送一个带有标识符 X 的请求.一旦 serviceB 完成 serviceA 要求的工作，它就会创建/添加到流(流特定于 X)一个新项目，让它知道它已经完成.

I have two micro services and I need to implement reliable notifications between them. I thought about using redis streams - serviceA will send a request to serviceB with an identifier X. Once serviceB is done doing the work serviceA asked for, it'll create/add to a stream (the stream is specific for X) a new item to let it know it's done.

ServiceA 可以发送多个请求，每个请求可能有不同的标识符.所以它会阻塞不同流中的新元素.

ServiceA can send multiple requests, each request may have a different identifier. So it'll block for new elements in different streams.

我的问题是如何删除不再需要的流，具体取决于它们的年龄.例如，我想删除一天前创建的流.这可能吗?

My question is how can I delete streams which are no longer needed, depending on their age. For example I'd like to have streams that were created over a day ago deleted. Is this possible?

如果不是，我很想听听您对如何在 redis 中不存在不需要的流的任何想法.

If it's not, I'd love to hear any ideas you have as to how not to have unneeded streams in redis.

谢谢

推荐答案

没有直接的方法可以根据 TTL/age 删除旧条目.您可以将 XTRIM/XDEL 与其他命令结合使用来修剪流.

There's no straight forward way to delete older entries based on the TTL/age. You can use a combination of XTRIM/XDEL with other commands to trim the stream.

让我们看看如何使用 XTRIM

XTRIM 流 MAXLEN ~ 大小

XTRIM stream MAXLEN ~ SIZE

XTRIM 将流修剪为给定数量的项目，并在需要时驱逐较旧的项目(ID 较低的项目).

XTRIM trims the stream to a given number of items, evicting older items (items with lower IDs) if needed.

您每天或根据您的删除策略定期生成流大小，并使用 XLEN 命令将其存储在某处

You generate the stream size every day or periodically based on your delete policy and store it somewhere using XLEN command

运行将调用 XTRIM 的定期作业

Run a periodic job that would call XTRIM as

XTRIM x-stream MAXLEN ~ (NEW_SIZE - PREVIOUS_SIZE)

例如，昨天的流大小是 500 现在是 600 然后我们需要删除 500 个条目以便我们可以运行

For example, yesterday stream size was 500 now it's 600 then we need to delete 500 entries so we can just run

XTRIM x-stream MAXLEN ~ 100

您可以使用不同的删除策略，例如每天、每周、每周两次等.

You can use different policies for deletion for example daily, weekly, twice a week, etc.

XDEL 流 ID [ID...]

XDEL stream ID [ID...]

从流中删除指定的条目，并返回删除的条目数，如果某些 ID 不存在，这可能与传递给命令的 ID 数不同.

Removes the specified entries from a stream, and returns the number of entries deleted, that may be different from the number of IDs passed to the command in case certain IDs do not exist.

所以您可以做的是，每当服务 B 使用该事件时，服务本身可以删除流条目，因为服务 B 知道流 ID，但是一旦您开始使用消费者组，这将无法工作.所以我会说使用 Redis 集或 Redis 映射来跟踪确认流 ID 并运行定期扫描作业来清理流.

So what you can do is whenever Service B consumes the event than the service itself can delete the stream entry as service B knows the stream ID, but this will not work as soon as you start using the consumer group. So I would say use Redis set or Redis map to track the acknowledge stream ids and run a periodic sweep job to clean up the stream.

例如

服务 A 向服务 B 发送一个 ID 为 1 的流项目服务 B 在消费地图中的项目后确认流项目ack_stream = { ID1: true }，您可以跟踪其他数据，例如以消费者组为例.

Service A sends a stream item with ID1 to service B Service B acknowledges the stream item after consuming the items in the map ack_stream = { ID1: true }, you can track other data e.g. count in case of the consumer group.

扫描作业将在每天凌晨 1 点左右定期运行，读取 ack_stream 的所有元素并过滤掉所有需要删除的项目.现在您可以使用一组流 ID 批量调用 XDEL 命令.

A sweep job would run at periodically like 1 AM daily that reads all the elements of ack_stream and filters out all items that require deletion. Now you can call XDEL commands in batch with the set of stream ids.

这篇关于如何为redis流定义TTL?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！