Hadoop WordCount示例 - 运行在Hadoop（Eclipse）选项不提示选择Hadoop服务器在窗口上运行 [英] Hadoop WordCount Example- Run On Hadoop(Eclipse) option is not prompting Select Hadoop server to run on window

问题描述

我试图在 Eclipse 上运行字数示例。通常当我们点击eclipse中的运行hadoop选项时，我们得到一个新窗口，要求选择服务器位置。但是，现在它直接运行程序，而不要求我从列表中选择一个现有的服务器。

我想是因为这样我得到以下异常：

  13/04/21 08:46:31 ERROR security.UserGroupInformation：PriviledgedActionException as：hduser1 cause：org.apache.hadoop.mapred.InvalidInputException：输入路径不存在：file：/home/hduser1/gutenbergIP/pg4300.txt 
线程main中的异常org.apache.hadoop.mapred.InvalidInputException：输入路径不存在：file：/home/hduser1/gutenbergIP/pg4300.txt 
  / pre> 
 
 
如果我从以下位置更改行，我的代码可以工作：
  FileInputFormat.setInputPaths（conf，/home/hduser1/gutenbergIP/pg4300.txt）; 
  
 to：
 code> FileInputFormat.setInputPaths（conf，hdfs：// localhost：54310 / home / hduser1 / gutenbergIP / pg4300.txt）; 
  
如果我明确指定了具有完整网址的文件名，它正在工作。我想要一些帮助。如何使我的相对URL工作，而不是提供完整的URL（必须在学校提交作为作业）。
解决方案
在您的代码中添加以下2行：
  config.addResource（new Path（/ HADOOP_HOME / conf / core-site。 XML））; 
 config.addResource（new Path（/ HADOOP_HOME / conf / hdfs-site.xml））; 
  
如果没有指定，您的客户端将查看本地FS，其中不包含指定的路径，因此抛出该错误。
 
I am trying to run word count example on Eclipse. Generally when we click on "run on hadoop" option in eclipse we get a new window asking to select server location. But, now it is directly running the program without asking me to choose an existing server from list below. 

I think because of this I am getting the following exception:
13/04/21 08:46:31 ERROR security.UserGroupInformation: PriviledgedActionException as:hduser1 cause:org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/home/hduser1/gutenbergIP/pg4300.txt
Exception in thread "main" org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: file:/home/hduser1/gutenbergIP/pg4300.txt
My code works if I change the line from:
FileInputFormat.setInputPaths(conf, "/home/hduser1/gutenbergIP/pg4300.txt");
to:
FileInputFormat.setInputPaths(conf, "hdfs://localhost:54310/home/hduser1/gutenbergIP/pg4300.txt");
If I explicitly specify the file name with full url it is working. I wanted some help regarding this. How can I make my relative URL work instead of giving a full URL(have to submit this as assignment in school).
解决方案
Add the following 2 lines in your code :
config.addResource(new Path("/HADOOP_HOME/conf/core-site.xml"));
config.addResource(new Path("/HADOOP_HOME/conf/hdfs-site.xml"));
If you don't specify this your client will look into the local FS, which doesn't contain the specified path, hence throwing that error.

                        
这篇关于Hadoop WordCount示例 - 运行在Hadoop（Eclipse）选项不提示选择Hadoop服务器在窗口上运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！