用于 Hadoop 的 DynamoDB 输入格式 [英] DynamoDB InputFormat for Hadoop

问题描述

我必须使用 Hadoop map reduce 处理一些保存在 Amazon Dynamo DB 中的数据.

I have to process some data which is persisted in Amazon Dynamo DB using Hadoop map reduce.

我在互联网上搜索 Dynamo DB 的 Hadoop InputFormat，但找不到.我不熟悉 Dynamo DB，所以我猜有一些与 DynamoDB 和 Hadoop 相关的技巧?如果有这种输入格式的实现，你能分享一下吗?

I was searching over internet for Hadoop InputFormat for Dynamo DB and couldn't find it. I'm not familiar with Dynamo DB so I'm guessing there is some trick related to DynamoDB and Hadoop? If there is anywhere implementation of this Input Format could you please share it?

推荐答案

经过大量搜索，我在 Amazon 的一个库中找到了 DynamoDBInputFormat 和 DynamoDBOutputFormat.

After a lot of searching I found DynamoDBInputFormat and DynamoDBOutputFormat in one of Amazon's libraries.

在 amazon elastic map reduce 上有一个名为 hive-bigbird-handler 的库，其中包含 dynamoDB 的输入和输出格式.完整的类名是:org.apache.hadoop.hive.dynamodb.write.DynamoDBOutputFormat 和 org.apache.hadoop.hive.dynamodb.read.DynamoDBInputFormat

On amazon elastic map reduce there is library called hive-bigbird-handler which contains input and output format for dynamoDB. Full class names are: org.apache.hadoop.hive.dynamodb.write.DynamoDBOutputFormat and org.apache.hadoop.hive.dynamodb.read.DynamoDBInputFormat

我希望这些课程对社区有用.

I hope these classes will be useful to community.

这篇关于用于 Hadoop 的 DynamoDB 输入格式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！