从Eclipse在AWS-EMR上运行MapReduce作业 [英] Running MapReduce jobs on AWS-EMR from Eclipse

问题描述

我在Eclipse中有WordCount MapReduce示例。
我将它导出到Jar，并将其复制到S3。然后我在AWS-EMR上运行它。成功。

I have the WordCount MapReduce example in Eclipse. I exported it to Jar, and copied it to S3. I then ran it on AWS-EMR. Successfully.

然后，我读了这篇文章 - http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html
显示如何使用AWS-EMR Api运行MapReduce作业。它仍然假设您的MapReduce代码打包在Jar中。

Then, I read this article - http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-common-programming-sample.html It shows how to use AWS-EMR Api to run MapReduce jobs. It still assumes your MapReduce code is packaged in a Jar.

我想知道是否有一种方法可以直接在AWS-EMR上运行来自Eclipse的MapReduce代码，必须将它导出到Jar。

I would like to know if there is a way to run MapReduce code from Eclipse directly on AWS-EMR, without having to export it to a Jar.

推荐答案

我还没有找到办法（对于用Java编写的mapreduce作业）。我相信没有Eclipse实用程序或插件可以做到这一点。
您可以编写某种脚本并将其合并到构建过程中，以将Jar推送到您的EMR集群。但是没有单一的实用程序/工具可以将mapreduce作业编译，捆绑，部署和执行到EMR。

I haven't found a way to do this (for mapreduce jobs written in Java). I believe there is no Eclipse utility or plugin that does this. You could maybe write a script of some kind and incorporate it in your build process to push out the Jar to your EMR cluster. But there is no single utility/tool that will compile, bundle, deploy and execute your mapreduce job to EMR.

这篇关于从Eclipse在AWS-EMR上运行MapReduce作业的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！