使用Hadoop 2.6.0在Windows上运行wordcount Hadoop示例 [英] Running wordcount Hadoop example on Windows using Hadoop 2.6.0

问题描述

我是Hadoop的新手，并了解到使用2.x版本，我可以在本地Windows 7 64位机器上试用Hadoop。

我安装了hadoop 2.6.0并安装了cygwin。 >
我可以执行bin / hadoop版本，但执行jar命令时出现以下错误：

注意：我还将winutils.jar放入bin中，从hadoop-common- 2.2.0.jar。
请帮忙。我无法摆脱这个错误。我还输入了输入和输出参数，但仍然失败。

  $ bin / hadoop jar /Hadoop/hadoop-2.6。 0 / share / hadoop / mapreduce / hadoop-mapreduce-examples-2.6.0.jar wordcount 
 15/02/03 12:40:45错误util.Shell：无法找到hadoop二进制路径中的winutils二进制文件
 java.io.IOException：无法在Hadoop二进制文件中找到可执行文件null\bin\winutils.exe。 
 at org.apache.hadoop.util.Shell.getQualifiedBinPath（Shell.java:355）
 at org.apache.hadoop.util.Shell.getWinUtilsPath（Shell.java:370）
在org.apache.hadoop.util.Shell。< clinit>（Shell.java:363）
 at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows 
（GenericOptionsParser.java:438）
 at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions 
（GenericOptionsParser.java:484）
 at org.apache.hadoop.util.GenericOptionsParser。< init> 
（GenericOptionsParser.java:170）
 at org.apache.hadoop.util.GenericOptionsParser。< init> 
（GenericOptionsParser.java:153）
 at org.apache.hadoop.examples.WordCount.main（WordCount.java:70）
 at sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method） 
 at sun.reflect.NativeMethodAccessorImpl.invoke 
（NativeMethodAccessorImpl.java:39）
 at sun.reflect.DelegatingMethodAccessorImpl.invoke 
（DelegatingMethodAccessorImpl.java:25）
在java.lang.reflect.Method.invoke（Method.java:597）
 at org.apache.hadoop.util.ProgramDriver $ ProgramDescription.invoke 
（ProgramDriver.java:71）
在org.apache.hadoop.util.ProgramDriver.run（ProgramDriver.java:144）
 at org.apache.hadoop.examples.ExampleDriver.main（ExampleDriver.java:74）
 at sun.reflect .NativeMethodAccessorImpl.invoke0（本地方法）
 at sun.reflect.NativeMethodAccessorImpl.invoke 
（NativeMethodAccessorImpl.java:39）
 at sun.reflect.DelegatingMethodAccessorImpl.invoke 
（DelegatingMethodAccessorImpl。 java：25）
 at java.lang.reflect.Method.invoke（Method.java:597）$ b $ at at or g.apache.hadoop.util.RunJar.run（RunJar.java:221）
 at org.apache.hadoop.util.RunJar.main（RunJar.java:136）
用法：wordcount<在> [< in>]< out> 
  

我也可以运行下面的命令：

$ $ $ $ $ $ $ $ $ $ / $看起来你已经安装了hadoop 2.6.0及以前版本的hadoop winutils。您必须安装您当前的hadoop版本的hadoop winutils。尝试从此github repo下载winutils https：// github。 com / steveloughran / winutils / tree / master / hadoop-2.6.0 / bin
最后用winutils bin目录替换你的bin目录！

I am new to Hadoop and learnt that with 2.x version, I can try Hadoop on my local Windows 7 64-bit machine.
I installed hadoop 2.6.0 and installed cygwin.
I could execute bin/hadoop version but I get the below error while executing the jar command:
Note: I have also placed the winutils.jar in the bin, from hadoop-common-2.2.0.jar.
Please help. I am not able to get rid of this error. I have also entered the input and output parameters, it still fails.

$ bin/hadoop jar /Hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount
15/02/03 12:40:45 ERROR util.Shell: Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows
(GenericOptionsParser.java:438)
at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions
(GenericOptionsParser.java:484)
at org.apache.hadoop.util.GenericOptionsParser.<init>
(GenericOptionsParser.java:170)
at org.apache.hadoop.util.GenericOptionsParser.<init>   
(GenericOptionsParser.java:153)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:70)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke
(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Usage: wordcount <in> [<in>...] <out>

I could run the below command as well:

$ bin/hadoop jar /Hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar

解决方案

It seems like that you have installed hadoop 2.6.0 and older version of hadoop winutils. You must install hadoop winutils of your current hadoop version. Try to download winutils from this github repo https://github.com/steveloughran/winutils/tree/master/hadoop-2.6.0/bin Finally replace your bin directory with the winutils bin directory!

这篇关于使用Hadoop 2.6.0在Windows上运行wordcount Hadoop示例的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！