具有可靠性的 Redis Pub/Sub [英] Redis Pub/Sub with Reliability
问题描述
我一直在考虑使用 Redis Pub/Sub 作为 RabbitMQ 的替代品.
I've been looking at using Redis Pub/Sub as a replacement to RabbitMQ.
根据我的理解，Redis 的 pub/sub 持有与每个订阅者的持久连接，如果连接终止，所有未来的消息都将丢失并丢弃在地板上.
From my understanding Redis's pub/sub holds a persistent connection to each of the subscribers, and if the connection is terminated, all future messages will be lost and dropped on the floor.
一种可能的解决方案是使用列表(和阻塞等待)来存储所有消息和发布/订阅作为通知机制.我认为这让我大功告成，但我仍然对失败案例有一些担忧.
One possible solution is to use a list (and blocking wait) to store all the message and pub/sub as just a notification mechanism. I think this gets me most of the way there, but I still have some concerns about the failure cases.
- 当订阅者死亡并重新上线时会发生什么?它应该如何处理所有待处理的消息?
- 当系统收到格式错误的消息时，您如何处理这些异常?死信队列?
- 是否有实施重试政策的标准做法?
推荐答案
当订阅者(消费者)死亡时，您的列表将继续增长，直到客户返回.一旦达到特定限制，您的生产者可以修剪列表(从任一侧)，但这是您需要在应用程序级别处理的事情.如果您在每条消息中都包含时间戳，那么您的使用者就可以根据消息的时长采取行动，假设您有要对消息时长强制执行的应用程序逻辑.
When a subscriber (consumer) dies, your list will continue to grow until the client returns. Your producer could trim the list (from either side) once it reaches a specific limit, but that is something you would need to handle at the application level. If you include a timestamp within each message, your consumer can then act on the age of a message, assuming you have application logic you want to enforce on message age.
我不确定格式错误的消息将如何进入系统，因为与 Redis 的连接通常是 TCP，并具有完整性保证.但是，如果发生这种情况，可能是由于生产者层消息编码中的错误，您可以通过保留接收消费者异常消息的每个生产者队列来提供处理错误的通用机制.
I'm not sure how a malformed message would enter the system, as the connection to Redis is usually TCP with the its integrity assurances. But if this happens, perhaps due to a bug in message encoding at the producer layer, you could provide a general mechanism for handling errors by keeping a queue-per-producer that received consumer's exception messages.
重试策略将在很大程度上取决于您的应用程序需求.如果您需要 100% 保证消息已被接收和处理，那么您应该考虑使用 Redis 事务 (MULTI/EXEC) 来包装消费者完成的工作，这样您就可以确保客户端不会删除消息，除非它已经完成了它的工作.如果您需要显式确认，那么您可以在专用于生产者进程的队列上使用显式 ACK 消息.
Retry policies will depend greatly on your application needs. If you need 100% assurance that a message has been received and processed, then you should consider using Redis transactions (MULTI/EXEC) to wrap the work done by a consumer, so you can ensure that a client doesn't remove a message unless it has completed its work. If you need explicit acknowlegement, then you could use an explicit ACK message on a queue dedicated to the producer process(es).
如果不了解您的应用需求，就很难知道如何明智地选择.通常，如果您的消息需要完整的 ACID 保护，那么您可能还需要使用 redis 事务.如果您的消息仅在及时时才有意义，则可能不需要交易.听起来好像您不能容忍丢弃的消息，因此您使用列表的方法很好.如果您需要为您的消息实现一个优先级队列，您可以使用排序集(Z 命令)来存储您的消息，使用它们的优先级作为分值，以及一个轮询消费者.
Without knowing more about your application needs, it's hard to know how to choose wisely. Generally, if your messages require full ACID protection, then you probably also need to use redis transactions. If your messages are only meaningful when they are timely, then transactions may not be needed. It sounds as though you can't tolerate dropped messages, so your approach of using a list is good. If you need to implement a priority queue for your messages, you can use the sorted set (the Z-commands) to store your messages, using their priority as the score value, along with a polling consumer.
这篇关于具有可靠性的 Redis Pub/Sub的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
