如何从亚马逊的弹性mapreduce（emr）群集上运行mapreduce作业？ [英] how to run a mapreduce job on amazon's elastic mapreduce (emr) cluster from windows?

问题描述

我试图学习如何在亚马逊的EMR上运行java Map / Reduce（M / R）作业。我所遵循的文档在这里 http://aws.amazon.com/articles/3938 。我在Windows 7电脑上。

当我尝试运行此命令时，显示帮助信息。

  ./ elasticmapreduce-client.rb RunJobFlow streaming_jobflow.json 
  

<当然，因为我在一台Windows机器上，我实际上输入了这个命令。我不知道为什么，但对于这个特定的命令，没有一个Windows版本（所有命令成对显示，一个用于* nix和一个用于Windows）。

 ruby​​ elastic-mapreduce RunJobFlow my_job.json 
  

我的问题是我们如何使用命令行界面（在Windows上）从Windows向亚马逊的EMR提交/运行作业？我试过在网上搜索，但我被带到狂野的地方。任何帮助表示感谢。

谢谢。 尝试使用--json选项。

例如 ./ elastic-mapreduce --create --name Multisteps --json wordcount_jobflow.json

您需要只用步骤修剪你的json文件（移除[]之外的所有内容）。有一个线索讨论： https://forums.aws.amazon.com/ thread.jspa？threadID = 35093

i'm trying to learn how to run a java Map/Reduce (M/R) job on amazon's EMR. the documentation that i am following is here http://aws.amazon.com/articles/3938. i am on a windows 7 computer.

when i try to run this command, i am shown the help information.

./elasticmapreduce-client.rb RunJobFlow streaming_jobflow.json 

of course, since i am on a windows machine, i actually type in this command. i am not sure why, but for this particular command, there was not a windows version (all commands where shown in pairs, one for *nix and one for windows).

 ruby elastic-mapreduce RunJobFlow my_job.json

my question is how do we submit/run a job from windows to amazon's EMR using the command line interface (on windows)? i've tried searching online, but i get taken to wild places. any help is appreciated.

thanks.

解决方案

Try using the --json option.

e.g. ./elastic-mapreduce --create --name Multisteps --json wordcount_jobflow.json

You will need to trim your json file with only the Steps (removing everything outside the []). There is a thread discussing that: https://forums.aws.amazon.com/thread.jspa?threadID=35093

这篇关于如何从亚马逊的弹性mapreduce（emr）群集上运行mapreduce作业？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！