简单的拉取消息队列 [英] Simple Pull Message Queue
问题描述
我正在努力寻找适合这项工作的工具.我探索了一些不同的消息队列，如 Kafka、Kestrel 等……我正在寻找具有 PULL 功能的东西.
I'm trying to find the right tool for the job. I've explored a few different message queues like Kafka, Kestrel, etc... and I'm looking for something that has a PULL functionality.
我有一个 API(分布式)可以将传入的消息推送到队列中.然后我让工作人员(单独的机器)从队列中提取消息.这确保了工作人员不会被淹没并且无法处理队列的负载.
I have an API (distributed) that shoves the incoming messages into the queue. I'd then have workers (separate machines) that pull messages from the queue. This ensures that the workers don't get flooded and can't handle the load of the queue.
我想知道 Kafka 或 Kestrel 是否支持这种类型的功能
推荐答案
Kafka 确实在推拉基础上工作，并且能够处理大规模实时流.同样如他们的文档中所述，Kafka 的性能在数据大小方面实际上是恒定的，因此保留大量数据不会成为问题.
Kafka does work on the push - pull basic and capable of handling large scale real time streams. Also as mentioned in their documentation Kafka's performance is effectively constant with respect to data size so retaining lots of data will not be a problem.
用于处理流结帐Storm.它是免费的、容错的、分布式的实时计算系统，并且非常容易扩展.它完全符合您的要求(在不同的机器上运行工人).并且它还支持 transactional 拓扑.最重要的是，它与 Apache Kafka 有很好的集成.
For processing stream Checkout Storm. Its free , fault-tolerant , distributed real time computation system and very easy to scale. It does what exactly you've mentioned (running workers on separate machines). And it also suppport transactional topologies. On top of that it has a very nice integration with Apache Kafka.
有关风暴的更多信息，请查看此处
For more on storm check here
因此，通常您可以做的是使用他们的消费 API 从 Kafka 队列中检索消息，然后将其提供给 Storm 集群，以分布式方式完成其余的工作.Kafka 0.8 提供了 2 种 API，
So typically what you can do is retrieve message from Kafka queue using their consume API and then feed it to a storm cluster to do the rest in a distributed manner. Kafka 0.8 provides 2 types of API,
高层或消费群低级或简单的消费者 API
前者为消费数据提供了高级抽象，并负责处理线程、错误处理等许多事情，而后者则允许对消息处理进行更好的控制，例如多次读取消息、消息事务等.
The former provides a high level abstraction for consuming data and takes care of lot of things like threading, error handling, while the later allows a much greater control over message handling like reading a message multiple times, message transaction etc.
这篇关于简单的拉取消息队列的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
