Avro 入门 [英] Getting Started with Avro

问题描述

我想开始使用 Avro 和 Map Reduce.有人可以建议一个很好的教程/示例来开始.我通过互联网搜索找不到太多内容.

I want to get started with using Avro with Map Reduce. Can Someone suggest a good tutorial / example to get started with. I couldnt find much through the internet search.

推荐答案

我最近做了一个项目，严重基于 Avro 数据，之前没有使用过这种数据格式，我不得不从头开始.你是对的，在开始使用 Avro 时很难从在线资源中获得很多帮助.我向您推荐的材料是:

I recently did a project that was heavily based on Avro data and not having used this data format before, I had to start from scratch. You are right in that it is rather hard to get much help from online sources when getting started with Avro. The material that I would recommend to you is:

• 到目前为止，我发现的最有用的资源是 Tom White 的 Hadoop 中的 Avro 部分 (p103-p116):The Definitive Guide 书以及他的 Github 页面他在书中使用的代码.
• 有关其他代码示例，我查看了 Ron Bodkin 的 Github 页面 avro-mr-sample.
• 在我的例子中，我使用 Python 来读写 Avro 文件，为此我使用了这个 教程.
• 尽管很明显，我还是会将链接添加到 Avro 用户邮件列表.在那里可以找到大量信息，在我阅读了上述材料并实现了一堆代码之后，我发现自己花了几个小时查看档案.

• By far, the most helpful source that I found was the Avro section (p103-p116) in Tom White's Hadoop: The Definitive Guide book as well as his Github page for the code he uses in the book.
• For additional code examples I looked at Ron Bodkin's Github page avro-mr-sample.
• In my case I used Python for reading and writing Avro files and for that I used this tutorial.
• Even though it is obvious, I will add the link to the Avro Users mailing list. There is a ton of information to be found there and after I had read the above material and implemented a bunch of code, I found myself spending hours looking through the archives.

最后，我给你的最后一个建议是使用 Avro 1.4.1 和 Hadoop 0.20.2 并且仅使用这种组合.我在使用 Hadoop 0.21 和更新的 Avro 版本运行代码时遇到了一些重大问题.

Finally, my last suggestion to you is to use Avro 1.4.1 with Hadoop 0.20.2 and ONLY that combination. I had some major issues getting my code to run using Hadoop 0.21 and more recent Avro versions.

这篇关于Avro 入门的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！