最简单的记录从Azure的事件集线器的所有邮件的方式 [英] Simplest way to log all messages from an Azure Event Hub

问题描述

我使用它输出到事件中心的服务。

I'm using a service which outputs to an Event Hub.

我们要存储输出，通过在Apache星火运行的批处理作业读取每天一次。基本上我们算了一下，一下就转储到斑点的所有消息。

We want to store that output, to be read once per day by a batch job running on Apache Spark. Basically we figured, just get all messages dumped to blobs.

什么是捕获从事件中心消息Blob存储最简单的方法？

我们的第一个想法是一个流媒体分析工作，但它需要解析的原始消息（CSV / JSON / Avro的），我们目前的格式是没有这些的。

Our first thought was a Streaming Analytics job, but it demands to parse the raw message (CSV/JSON/Avro), our current format is none of those.

的更新的我们改变我们的信息格式，解决了这个问题。我还是想知道是否有任何影响较小的方式来存储邮件到斑点。难道EventHub有一个解决方案之前，流媒体分析来了吗？

Update We solved this problem by changing our message format. I'd still like to know if there's any low-impact way to store messages to blobs. Did EventHub have a solution for this before Streaming Analytics arrived?

推荐答案

您可以编写自己的工作进程读出EventHub的消息，并将其存储到Blob存储。你并不需要这样做实时作为EH的消息仍然是一组保留天数。读取的EH客户负责管理的是什么讯息已通过跟踪EH消息的partitionid的处理和偏移。有一个C＃库，使这个非常容易和尺度真的很好：<一href=\"https://azure.microsoft.com/en-us/documentation/articles/event-hubs-csharp-ephcs-getstarted/\">https://azure.microsoft.com/en-us/documentation/articles/event-hubs-csharp-ephcs-getstarted/

You could write your own worker process to read the messages off EventHub and store them to blob storage. You do not need to do this real time as messages on EH remain for the set retention days. The client that reads the EH is responsible for managing what messages have been processed by keeping track of the EH message partitionid and offset. There is a C# library that makes this extremely easy and scales really well: https://azure.microsoft.com/en-us/documentation/articles/event-hubs-csharp-ephcs-getstarted/

这篇关于最简单的记录从Azure的事件集线器的所有邮件的方式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！