hadoop fs -ls 导致“没有这样的文件或目录"； [英] hadoop fs -ls results in "no such file or directory"
问题描述
我已经为 10 节点集群安装并配置了 Hadoop 2.5.2.1 作为主节点，其他节点作为从节点.
我在执行 hadoop fs 命令时遇到问题.hadoop fs -ls 命令与 HDFS URI 一起工作正常.在没有 HDFS URI 的情况下使用时，它会给出消息ls: `.': No such file or directory"
ubuntu@101-master:~$ hadoop fs -ls15/01/30 17:03:49 警告 util.NativeCodeLoader:无法加载本机 Hadoop适用于您平台的图书馆...在适用的情况下使用内置 Java 类ls: `.': 没有那个文件或目录ubuntu@101-master:~$然而，使用 HDFS URI 执行相同的命令
ubuntu@101-master:~$ hadoop fs -ls hdfs://101-master:50000/15/01/30 17:14:31 警告 util.NativeCodeLoader:无法加载本机 Hadoop适用于您平台的库...在适用的情况下使用内置 Java 类找到 3 个项目drwxr-xr-x - ubuntu 超级组 0 2015-01-28 12:07 hdfs://101-master:50000/hvision-data-rw-r--r-- 2 ubuntu 超级组 15512587 2015-01-28 11:50 hdfs://101-master:50000/testimage.seqdrwxr-xr-x - ubuntu 超级组 0 2015-01-30 17:03 hdfs://101-master:50000/wrodcount-inubuntu@101-master:~$由于这种行为，我在 MapReduce 程序中遇到异常.jarlib 指的是 HDFS 文件位置，而我希望 jarlib 指的是存储在 Hadoop 节点上的本地文件系统中的 jar 文件.
您所看到的行为是意料之中的，让我解释一下当您使用 hadoop fs 命令时会发生什么.
命令的语法是这样的:hadoop fs -ls [path]
默认情况下，当你没有为上述命令指定[path]时，hadoop将路径扩展到hdfs中的/home/[username]；其中 [username] 被替换为正在执行命令的 linux 用户名.
所以，当你执行这个命令时:
ubuntu@xad101-master:~$ hadoop fs -ls你看到错误的原因是 ls: '.': No such file or directory 因为 hadoop 正在寻找这个路径 /home/ubuntu，它似乎就像这个路径在hdfs中不存在.
这个命令的原因:
ubuntu@101-master:~$ hadoop fs -ls hdfs://101-master:50000/ 正在工作，因为您已明确指定 [path] 并且是 hdfs 的根.你也可以用这个来做同样的事情:
ubuntu@101-master:~$ hadoop fs -ls/它会自动评估为 hdfs 的根.
希望，这会清除您在执行 hadoop fs -ls 命令时看到的行为.
因此，如果要指定本地文件系统路径，请使用 file:/// url scheme.
I have installed and configured Hadoop 2.5.2 for a 10 node cluster. 1 is acting as masternode and other nodes as slavenodes.
I have problem in executing hadoop fs commands. hadoop fs -ls command is working fine with HDFS URI. It gives message "ls: `.': No such file or directory" when used without HDFS URI
ubuntu@101-master:~$ hadoop fs -ls
15/01/30 17:03:49 WARN util.NativeCodeLoader: Unable to load native-hadoop
ibrary for your platform... using builtin-java classes where applicable
ls: `.': No such file or directory
ubuntu@101-master:~$
Whereas, executing the same command with HDFS URI
ubuntu@101-master:~$ hadoop fs -ls hdfs://101-master:50000/
15/01/30 17:14:31 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x - ubuntu supergroup 0 2015-01-28 12:07 hdfs://101-master:50000/hvision-data
-rw-r--r-- 2 ubuntu supergroup 15512587 2015-01-28 11:50 hdfs://101-master:50000/testimage.seq
drwxr-xr-x - ubuntu supergroup 0 2015-01-30 17:03 hdfs://101-master:50000/wrodcount-in
ubuntu@101-master:~$
I am getting exception in MapReduce program due to this behavior. jarlib is referring to the HDFS file location, whereas, I want jarlib to refer to the jar files stored at the local file system on the Hadoop nodes.
The behaviour that you are seeing is expected, let me explain what's going on when you are working with hadoop fs commands.
The command's syntax is this: hadoop fs -ls [path]
By default, when you don't specify [path] for the above command, hadoop expands the path to /home/[username] in hdfs; where [username] gets replaced with linux username who is executing the command.
So, when you execute this command:
ubuntu@xad101-master:~$ hadoop fs -ls
the reason you are seeing the error is ls: '.': No such file or directory because hadoop is looking for this path /home/ubuntu, it seems like this path doesn't exist in hdfs.
The reason why this command:
ubuntu@101-master:~$ hadoop fs -ls hdfs://101-master:50000/
is working because, you have explicitly specified [path] and is the root of the hdfs. You can also do the same using this:
ubuntu@101-master:~$ hadoop fs -ls /
which automatically gets evaluated to the root of hdfs.
Hope, this clears the behaviour you are seeing while executing hadoop fs -ls command.
Hence, if you want to specify local file system path use file:/// url scheme.
这篇关于hadoop fs -ls 导致“没有这样的文件或目录"；的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
