Redis 作为消息代理 [英] Redis as a message broker

问题描述

我想以发布-订阅方式在应用程序之间传递数据.数据产生的速度可能比消耗的速度快得多，而且消息会丢失，这不是问题.想象一个快速传感器和一个慢速传感器数据处理器.为此，我使用 redis pub/sub 并编写了一个类作为订阅者，接收每条消息并将其放入缓冲区.当真实"函数请求消息时，缓冲区会被覆盖或无效.所以当我问这个类时，我立即得到了回应(暗示我的函数比数据进来的慢)或者我必须等待(暗示我的函数比数据快).

I want to pass data between applications, in a publish-subscribe manner. Data may be produced at a much higher rate than consumed and messages get lost, which is not a problem. Imagine a fast sensor and a slow sensor data processor. For that, I use redis pub/sub and wrote a class which acts as a subscriber, receives every message and puts that into a buffer. The buffer is overwritten when a new message comes in or nullified when the message is requested by the "real" function. So when I ask this class, I immediately get a response (hint that my function is slower than data comes in) or I have to wait (hint that my function is faster than the data).

这对于数据快速传入的情况非常有效.但是对于相对很少出现的数据，假设每五秒，这不起作用:想象我的消费者在生产者之后稍微启动，第一条消息丢失并且我的消费者需要等待近 5 秒钟才能开始工作.

This works pretty good for the case that data comes in fast. But for data which comes in relatively seldom, let's say every five seconds, this does not work: imagine my consumer gets launched slightly after the producer, the first message is lost and my consumer needs to wait nearly five seconds, until it can start working.

我想我必须用Redis工具来解决这个问题.而不是pub/sub，我可以简单地使用get/set 方法，从而将缓存功能直接放入Redis.但是，我的消费者将不得不轮询数据库，而不是我目前拥有的事件魔法.密钥可能看起来像key:timestamp"，我的消费者现在必须get key:* 并永久比较时间戳，我认为这会导致大量负载.自然没有睡眠的可能性，因为虽然我不关心丢失的消息(我无能为力)，但我确实关心延迟.

I think I have to solve this with Redis tools. Instead of a pub/sub, I could simply use the get/set methods, thus putting the cache functionality into Redis directly. But then, my consumer would have to poll the database instead of the event magic I have at the moment. Keys could look like "key:timestamp", and my consumer now has to get key:* and compare the timestamps permamently, which I think would cause a lot of load. There is no natural possibility to sleep, since although I don't care about dropped messages (there is nothing I can do about), I do care about delay.

是否有人将 Redis 用于类似的事情，并且可以给我一些有关如何巧妙使用 Redis 工具和数据结构的提示?

Does someone use Redis for a similar thing and could give me a hint about clever use of Redis tools and data structures?

编辑

理想情况下，我的程序流程如下所示:

Ideally, my program flow would look like this:

• 启动程序
• 从Redis中检索key
• 告诉 Redis，嘿，通知我 key 的变化".
• 异步启动一些东西，并回调新消息.

写到这里，想到了一个想法:发布者不仅在主题key上发布message，还发布set key message.这样，应用程序可以先get，然后subscribe.

By writing this, an idea came up: The publisher not only publishes message on topic key, but also set key message. This way, an application could initially get and then subscribe.

好主意与否?

Keyspace 通知正是我所需要的.Redis 作为信息的主要来源，我的客户端订阅键空间通知，它会通知订阅者有关影响特定键的事件.现在，在我的客户端的异步部分，我订阅了有关我感兴趣的密钥的通知.这些通知设置了 key_has_updates 标志.当我需要该值时，我从 Redis get 它并取消设置标志.使用未设置的标志，我知道服务器上没有该键的新值.如果没有键空间通知，这将是我需要轮询服务器的部分.优点是我可以使用各种数据结构，不仅是pub/sub机制，还有一个错过第一个事件的slow joiner总是能够get 初始值，使用 pub/sib 会丢失.

Keyspace notifications are really what I need here. Redis acts as the primary source for information, my client subscribes to keyspace notifications, which notify the subscribers about events affecting specific keys. Now, in the asynchronous part of my client, I subscribe to notifications about my key of interest. Those notifications set a key_has_updates flag. When I need the value, I get it from Redis and unset the flag. With an unset flag, I know that there is no new value for that key on the server. Without keyspace notifications, this would have been the part where I needed to poll the server. The advantage is that I can use all sorts of data structures, not only the pub/sub mechanism, and a slow joiner which misses the first event is always able to get the initial value, which with pub/sib would have been lost.

当我需要该值时，我从Redis 获取该值并将标志设置为false.

When I need the value, I obtain the value from Redis and set the flag to false.

推荐答案

一个想法是将数据推送到一个列表 (LPUSH) 并修剪它(LTRIM)，所以它不会永远增长，如果有没有消费者.另一方面，消费者将从该列表中获取项目并对其进行处理.您还可以使用 keyspace 通知，并在每次将项目添加到该队列时收到提醒.

One idea is to push the data to a list (LPUSH) and trim it (LTRIM), so it doesn't grow forever if there are no consumers. On the other end, the consumer would grab items from that list and process them. You can also use keyspace notifications, and be alerted each time an item is added to that queue.

这篇关于Redis 作为消息代理的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！