Amazon EC2 与 Amazon EMR [英] Amazon EC2 vs. Amazon EMR

问题描述

我在 Hive 中实现了一个任务.目前它在我的单节点集群上运行良好.现在我计划在 AWS 上部署它.

I have implemented a task in Hive. Currently it is working fine on my single node cluster. Now I am planning to deploy it on AWS.

我对 AWS 一无所知.如果我打算部署它，我应该选择 Amazon EC2 还是 Amazon EMR?

I don't know anything about the AWS. If I plan to deploy it then what should I choose Amazon EC2 or Amazon EMR?

我想提高我的任务的表现.哪一个对我来说更好更可靠?如何接近他们?我听说我们也可以像在 AWS 上一样注册我们的 VM 设置.可能吗?

I want to improve the performance of my task. Which one is better and reliable for me? How to approach towards them? I heard that we can also register our VM setting as it is on AWS. Is it possible?

请尽快给我推荐.

非常感谢.

推荐答案

EMR 是 EC2 实例的集合，其中安装并配置了 Hadoop(以及可选的 Hive 和/或 Pig).如果您使用集群来运行 Hadoop/Hive/Pig 作业，则 EMR 是您的最佳选择.与 EC2 实例相比，EMR 实例的成本略高.今天对亚马逊价格的快速检查显示，小型 EC2 实例的成本为 0.08 美元/小时，而小型 EMR 实例的成本为 0.015 美元/小时.在我看来，为了省去安装和设置 Hadoop(以及 Hive 和 Pig)、创建和维护 AMI 以及使用它的麻烦，支付额外的钱是完全值得的.此外，EMR 的 Hadoop 和 Hive 版本有一些补丁在 Apache Hive 上不可用(至少目前还没有).如果您使用 EC2，您可能会使用 Apache Hadoop 和 Hive(或者可能是 cloudera 发行版)并且无法访问这些补丁(例如对 S3 的本机支持或诸如 ALTER TABLE my_table RECOVER PARTITIONS

EMR is a collection of EC2 instances with Hadoop (and optionally Hive and/or Pig) installed and configured on them. If you are using your cluster for running Hadoop/Hive/Pig jobs, EMR is the way to go. An EMR instance costs a little bit extra as compared to an EC2 instance. A quick check on Amazon prices today reveals that a small EC2 instances costs $0.08/hour while a small EMR instance costs $0.015/hour extra. In my opinion, it's totally worth paying that extra money to save yourself the hassle of installing and setting up Hadoop (along with Hive and Pig), creating and maintaining and AMI and using it. Moreover, EMR's version of Hadoop and Hive has some patches that are not available (atleast, not yet) on Apache Hive. If you use EC2, you will probably be using Apache Hadoop and Hive (or may be, the cloudera distributions) and wouldn't have access to those patches (like native support for S3 or commands like ALTER TABLE my_table RECOVER PARTITIONS

参考文献:

• http://aws.amazon.com/ec2/pricing/
• http://aws.amazon.com/elasticmapreduce/pricing/

这篇关于Amazon EC2 与 Amazon EMR的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！