如何在Cloudfoundry上使用Kafka和Storm? [英] How to use kafka and storm on cloudfoundry?
问题描述
我想知道是否可以将kafka作为云原生应用程序运行，我是否可以在Pivotal Web Services上创建kafka集群作为服务.我不想只集成客户端，我想运行kafka集群/服务本身吗?
谢谢， 尼尔
我可以为您提供一些起点，从这些起点到功能齐全的地方将涉及一些工作.
一种选择是使用docker映像在Cloud Foundry(例如Pivotal Web Services)上部署kafka集群. Spotify具有Docker化的kafka和kafka-proxy (包括Zookeeper).要记住的一件事是，PWS当前不支持具有持久性的应用程序(尽管 kafka BOSH版本.
一旦集群运行，就有两种方法将Cloud Foundry应用程序与其集成在一起.最简单的就是将其作为"用户提供的服务"，可让您将kafka群集访问信息传递到您的应用程序.另一种选择是将服务代理放置在您的群集之前，如果您有许多不同的人将推送需要与kafka群集通信的应用程序，这将特别有用.不必每次都手动告诉人们访问信息，他们可以做一些简单的事情，例如cf bind-service SOME_APP YOUR_KAFKA_SERVICE.这是一个 kafka服务经纪人和 解决方案
I can point you at a few starting points, there would be some work involved to go from those starting points to something fully functional.
One option is to deploy the kafka cluster on Cloud Foundry (e.g. Pivotal Web Services) using docker images. Spotify has Dockerized kafka and kafka-proxy (including Zookeeper). One thing to keep in mind is that PWS currently doesn't support apps with persistence (although this work is starting) so if you were to go this route right now, you would lose the data in kafka when the application is rolled. Looking at that Spotify repo, it looks like the docker images are generally run without any mounted volumes, so this persistence-less kafka seems like it may be a valid use case (I don't know enough about kafka to say).
The other option is to deploy kafka directly on some IaaS (e.g. AWS) using BOSH. BOSH can be hard if you're seeing it for the first time, but it is the ideal way to deploy any distributed software that you want running on VMs. You will also be able to have persistent volumes attached to your kafka VMs if necessary. Here is a kafka BOSH release which may work.
Once you have your cluster running, you have two ways to integrate your Cloud Foundry applications with it. The simplest is just to provide it to your applications as a "user-provided service", which lets you flow kafka cluster access info to your apps. The alternative would to put a service broker in front of your cluster, which would be especially useful if you have many different people who will be pushing apps that need to talk to the kafka cluster. Rather than you having to manually tell people the access info each time, they can do something simple like cf bind-service SOME_APP YOUR_KAFKA_SERVICE. Here is a kafka service broker along with more info about service brokers in general.
这篇关于如何在Cloudfoundry上使用Kafka和Storm?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
